The School Interference Questionnaire: Examining a Novel Measure of Mental Health–Related Academic Functional Impairment in Youth

Abstract

This article reports on the development and initial psychometric properties of the School Interference Questionnaire (SIQ), a questionnaire designed to assess both academic functional impairment related to mental health problems and the type and frequency of school refusal behavior. Participants were 110 youth ages 13 to 18 years (M = 15.41 years, SD = 1.42) admitted to an adolescent partial hospitalization program. The majority of participants identified as female (57.3%), Caucasian (59.1%), and non-Latino (70.0%). Internal consistency of the 12 SIQ core interference items was excellent (Cronbach’s α = .91). The factor structure suggested that a single factor was appropriate for the 12 school interference items. Correlations between the SIQ and other measures in this study provided support for construct validity. The average SIQ core interference item score showed evidence of convergent validity via correlations with measures of school refusal, global disability, and internalizing symptoms. The SIQ also exhibited no significant association with timing of study enrollment, suggesting evidence for divergent validity. Results provide initial evidence supporting the psychometric properties of this novel measure.

Keywords

assessment child mental health school/classroom academic behaviors academic achievement/failure

School absences are related to a number of mental health problems, as well as a number of long-term negative outcomes, including serious psychiatric, health, and legal problems (Kearney, 2008). Avoidance of school and school-related activities can interrupt growth in key areas of social, cognitive, and emotional development (Knollmann et al., 2010; Sato et al., 2007). In fact, the assessment of functioning in school is key to a full understanding of youth development, and problems in this domain can contribute to a cascade of problems in other areas of youth development (e.g., family relationships, peer relationships, self-concept and self-esteem, and cognitive development). Too often, difficulties in these key domains of functioning are excluded or overlooked in the assessment of Diagnostic and Statistical Manual of Mental Disorders (5th ed.; DSM-5; American Psychiatric Association, 2013) criteria for psychiatric disorders (Fabiano & Pelham, 2016; Gadow et al., 2013; Rapee, 2012). Assessing and addressing functional impairment is necessary for improving the developmental trajectory of youth. Measuring academic functional impairment may be especially important now because stressors related to the COVID-19 pandemic and its impact on youth, families, and schooling could make mental health problems, school anxiety, and school refusal even more prevalent.

There is a paucity of literature on how extensive school interference is among youth with mental health problems. Little is known about the frequency, type, and intensity of missed school in this population (Becker et al., 2014). Extant assessment tools do not fully capture the frequency and intensity of school refusal, nor the extent of academic functional impairment beyond school absences. Academic goals, ability to complete schoolwork and homework, school achievement, late arrivals and/or early dismissals, time spent outside of the classroom during the school day (e.g., in the nurse’s office), and how one spends their time during the school day while they are not attending school are all central to a full understanding of school interference related to psychiatric symptoms. This article describes the development and preliminary psychometric properties of a novel measure of school interference that fills a gap in the current assessment landscape.

Existing Measures

The current menu of school refusal measures is either too broad to capture the depth of academic functional impairment or too narrow to capture its breadth. For example, the World Health Organization Disability Assessment Schedule (WHODAS) 2.0 is a 12-item scale with self-report and proxy-report (e.g., caregivers) versions that measures disability related to physical and mental health problems (Andrews et al., 2009; Rehm et al., 1999). While this is a useful measure of overall impairment, it does not capture details about specific areas of academic functional impairment. Similarly, the Work and Social Adjustment Scale for Youth (WSASY) is a five-item, self-report measure of psychosocial impairment in youth, regardless of mental health status (De Los Reyes et al., 2019). While this is a useful and youth-specific measure for capturing various elements of impairment across youth’s lives, it includes only one item addressing impairment related to school and therefore cannot capture the multifaceted ways mental health problems may get in the way at school.

One of the few existing measures that specifically assess school refusal is the School Refusal Assessment Scale–Revised (SRAS-R; Kearney, 2002; Kearney & Silverman, 1993). This measure is a useful clinical and research tool, with child- and parent-report versions, that has created a rubric for understanding school refusal behavior and the various functions underlying this diverse construct. Following the development of the SRAS, Kearney and colleagues created and confirmed via factor analysis four functional profiles of school refusal: (a) avoidance of school-based stimuli that provoke negative affectivity, (b) escape from aversive social and/or evaluative situations, (c) pursuit of attention from significant others, and/or (d) pursuit of tangible reinforcers outside of school (Kearney, 2003, 2006; Kearney & Albano, 2004; Kearney & Silverman, 1996). They found that negatively reinforced school refusal behavior (Functions 1 and 2) was largely associated with internalizing symptoms and diagnoses, and positively reinforced school refusal behavior (Functions 3 and 4) was largely associated with externalizing symptoms and diagnoses (Kearney & Silverman, 1993).

Despite the tremendous contribution of the SRAS, the measure does not fully capture the degree of academic or other functional impairments related to mental health problems (e.g., impacts on grades, concentration, and completion of classwork and homework). This instrument also does not assess the type, frequency, or intensity of school refusal. Thus, there is a need for a measure that assesses the frequency, type, and intensity of school refusal alongside the impact of mental health problems on the varied areas of school-related functioning. A measure of academic functional impairment could complement existing measures and provide (a) information about the impact of mental health problems on various areas of academic functioning, including goals, ability to complete homework, and so on, (b) specific information about how frequent and pervasive school refusal is for a particular patient or student, (c) information about degree or form of school refusal (e.g., not coming into school at all vs. “frequent fliers” in the school counselor or nurse’s office), (d) description of how youth are spending their time at home during school hours, and (e) a retrospective assessment of the progression of school refusal behavior. Each of the five areas of information listed above could help inform targets for intervention. For example, clinicians could use such a measure to inform behavior plans for youth who stay at home and play video games, inform parental guidance and recommendations (e.g., not enabling avoidance by picking up youth early from school), design exposure therapy exercises targeting tests or homework, and/or select coping skills to aid in staying in the classroom or taking tests.

Assessment Within Intensive Treatment Settings

The clinical utility of a measure with the attributes listed above is particularly salient in an acute care setting where school refusal and academic functional impairment are common. Youth with greater impairment are more likely to be referred to more intensive treatment settings (McDermott et al., 2002), and youth enrolled in acute ambulatory psychiatric settings (e.g., partial hospitalization programs [PHPs] and day treatment) are more likely to experience a pervasive disruption in school attendance and related areas of academic performance. Thus, a measure that can efficiently assess the function, frequency, types, and progression of school refusal behavior can optimize a treatment team’s capacity to quickly develop a personalized treatment plan and intervene so that youth can rapidly transition from acute care back into school. Further understanding the phenomenology of school refusal behavior in acute care settings, like a PHP, will likely be clinically useful for care providers working in these settings and substantially add to the scarce literature on PHP day treatment program settings (Kennair et al., 2011; Thatte et al., 2013).

Study Aims

In this study, we aimed to (a) expand the knowledge base about school-related impairment by developing a child- and parent-report measure to assess school interference related to mental health problems in youth that captures the frequency, type, and intensity of missed school; and (b) examine the initial psychometric properties of the youth-report version of the measure. Psychometric properties were evaluated by evaluating internal consistency and construct validity (convergent and divergent validity) of the youth-report version of the measure, as well as parent–child agreement.

Method

Participants

Participants were 110 youth ages 13 to 18 years (M = 15.41 years, SD = 1.42) who were admitted to an adolescent PHP located in the New York City metropolitan area. Caregivers (N = 98) of the youth also participated. More than half of the youth participants (57.3%) identified as female, 36.4% identified as male, and 6.4% self-identified in gender nonbinary categories. Youth or caregiver report of race and ethnicity was collected. Slightly more than half of the sample (59.1%) identified as Caucasian. The remaining participants identified as more than one race (12.7%), Asian (8.2%), African American (8.2%), American Indian (<1%), Other (5.5%), and 5.5% did not identify a race. Nearly two thirds (70.0%) of participants identified as non-Hispanic/Latino. The remaining participants identified as Hispanic/Latino (22.7%) or did not respond to this item (7.3%). Caregiver participants included mostly mothers (60.2%), followed by fathers (17.3%), mothers and fathers (12.2%), and other (2.0%); 8.2% of caregivers did not specify the relationship. More than half of the primary or secondary caregivers (54.5%) completed a postgraduate degree, followed by college or technical/trade school (34.5%) and high school or General Educational Development (GED; 7.3%); 3.6% of caregivers did not report on the education level.

Diagnostic data were extracted from the PHP discharge summary in the participant’s electronic medical record. Diagnoses therefore reflect the interdisciplinary treatment team’s diagnostic impressions at the end of the participant’s stay in the program, which were informed by the team’s initial clinical assessment, individual and family sessions, collateral information, and direct observations of the youth during the course of treatment within the PHP. Youth participants in this sample represented a broad range of DSM-5 diagnoses (American Psychiatric Association, 2013; see Table 1 for participants’ diagnoses). Comorbidity was the norm among the participants, as 70.0% had more than one diagnosis, 46.4% had two diagnoses, 17.3% had three diagnoses, and 6.4% had four diagnoses.

Table 1.

Frequencies (n) and Percentages of Participants’ Diagnoses.

Diagnosis	n	%
Major depressive disorder	77	70.0
Generalized anxiety disorder	30	27.3
Social anxiety disorder	24	21.8
Unspecified anxiety disorder	24	21.8
Attention-deficit/hyperactivity disorder	15	13.6
Obsessive compulsive and related disorders	12	10.9
Bipolar disorder	8	7.3
Panic disorder	6	5.5
Substance use disorder	5	4.5
Post-traumatic stress disorder	3	2.7
Schizophrenia spectrum and other psychotic disorders	3	2.7
Separation anxiety disorder	2	1.8
Adjustment disorders	2	1.8
Gender dysphoria	2	1.8
Oppositional defiant disorder	2	1.8
Autism spectrum disorder	1	0.9

Note. Some participants had multiple diagnoses.

Procedures

All youth who were admitted to the adolescent PHP with at least one English-speaking caregiver were invited to participate in this study, which was part of a larger repository study. Youth and their caregivers were approached for the study during the course of their stay in the PHP. Unless the youth was 18 years old, youth participants were included in the study only when both the caregiver signed consent and the youth provided assent to participate. Of the 263 youth who were admitted between October 18, 2016, and December 31, 2018, 144 (54.75%) consented to participate in this study. The remaining 119 youth did not participate for the following reasons: 64 youth (53.8%) were discharged before both the youth and the caregiver supplied consent to the study, 43 youth or their caregivers (36.1%) declined, and 12 (10.1%) did not have English-speaking caregivers and therefore could not provide consent. The most common reason for declining to participate in the study was due to privacy concerns (34.9%). Of the 144 youth who had consent, 110 youth completed at least 50% of the School Interference Questionnaire (SIQ) measure items and were included in the sample for this study.

All youth and their caregivers were given a battery of questionnaires on the day of the intake as part of standard clinical care within the PHP. Families completed these questionnaires on site using paper and pencil. A research assistant was available to answer participant questions about the measures. When participants consented to participate in this study, these intake questionnaires, as well as clinical information from the electronic health record, were retained and used for study purposes. No incentives were given for participation. Study procedures were approved by the authors’ institutional review board and conducted in accordance with ethical standards.

Measures

SIQ development

Development of the SIQ involved several steps. First, items were generated and formatted by the investigators to be relevant for adolescents and their caregivers with respect to content and language. The SIQ items were designed with an aim to capture academic functional impairment of youth struggling with mental health problems across a wide range of diagnostic categories and levels of symptom severity. Items were selected based on clinical experience and literature review (e.g., American Psychiatric Association, 2013; Epstein, 1999; Goldstein & Naglieri, 2016; Hoagwood et al., 1996; John et al., 1987; Langley et al., 2004, 2014; Park et al., 2011; Sawyer et al., 2000; Winters et al., 2005). The measure was designed with the intention of capturing the multifaceted ways that mental health problems affect school, including attendance (i.e., “staying in the classroom”), concentration (i.e., “concentrating on schoolwork”), completion of school-related work (i.e., “completing work in class”), current academic achievement (i.e., “getting the grades you want”), and future goal attainment (i.e., “goals you have for the future”).

Second, the preliminary SIQ was reviewed by a team of six child psychologists and three child and adolescent psychiatrists with expertise in treating school refusal. Feedback was solicited about the various ways in which youth can experience school interference due to mental health disorders to ensure that items captured the multifaceted ways in which school can be affected. Feedback was also requested about item content, wording, and formatting used in the measure. Feedback was integrated into a revised version of the measure that was rereviewed by the same group of individuals for additional comments.

Third, the measure was pilot tested using a sample of adolescents and their parents who were admitted to the PHP in which this study was performed. These pilot data, which are not included in this study, informed the feasibility of using the SIQ as a self-report questionnaire with youth seeking treatment from an acute level of care. Client feedback about item content, wording, and practical administration was received and integrated into the version of the questionnaire used for this study.

There are youth- and caregiver- report versions of the measure. Twelve core SIQ items assess areas of academic functional impairment by having respondents rate the extensiveness of impairment in each domain. Each item begins with the stem, “How much have anxiety, depression, or other mental health problems gotten in the way of . . .,” and each item lists a potential area of impairment. Youth and caregivers rate the extent of interference for each item using a 5-point Likert-type scale, with items ranging from 0 (not at all) to 4 (very much). See Table 2 for specific items.

Table 2.

School Interference Questionnaire Core Interference Average Item Scores.

Item	M	SD	Frequency of endorsement
Item	M	SD	0	1	2	3	4
1. . . . goals you have for the future	2.99	1.31	10	4	16	20	53
2. . . . your ability to apply for college	1.42	1.56	42	8	13	12	14
3. . . . getting the grades you want	2.65	1.36	9	14	23	16	42
4. . . . concentrating on schoolwork	2.86	1.22	6	11	14	31	40
5. . . . staying in the classroom	1.86	1.54	30	18	16	17	23
6. . . . setting educational goals	2.49	1.46	16	12	18	21	37
7. . . . studying	2.79	1.31	8	11	20	21	44
8. . . . completing your homework	2.69	1.35	13	7	16	31	37
9. . . . completing work in class	2.22	1.48	19	17	18	20	29
10. . . . being enrolled in the school you would like	1.54	1.58	42	11	18	9	20
11. . . . your scores on standardized tests	1.82	1.64	37	10	15	14	26
12. . . . taking a full course load	2.10	1.68	29	11	11	13	33
Average overall interference	2.31	1.06	—	—	—	—	—

Note. All items begin with “How much have anxiety, depression, or other mental health problems gotten in the way of . . .” Items are scored on a scale 0 (not at all) to 4 (very much). Range for each item was 0 to 4.

This measure also includes additional items, presented in a forthcoming manuscript, that assess the “type” (e.g., commonly avoided parts of the school day) and intensity of school refusal in the past week and month. For each time frame, participants indicate how often they missed full days, arrived late, left early, and left the classroom due to mental health problems using a 5-point Likert-type scale with the following anchors: 0 (none), 1 (a little), 2 (some), 3 (much), and 4 (all). The SIQ also contains items to help clinicians assess how and where youth spend their time when they stay home during school hours. Finally, items were included to indicate the number of days missed in the past week and month due to psychiatric hospitalization (e.g., “In the past year, how many days were you hospitalized on an inpatient unit?”). For a more detailed overview of this, see Rohrig et al. (2020, 2021). Items are identical on the youth- and caregiver-report versions of the SIQ, with the exception that the caregiver version includes seven additional yes/no, forced choice, or qualitative items intended to help clinicians gather information about the child’s school placement and special services (e.g., Individualized Education Plan, 504 plan). This article focuses on examining the psychometric properties of the youth-report version of the SIQ.

After the item generation and refinement stage of the scale development was complete, the authors reviewed the reliability and factor structure of the 12 core items, which informed a scoring strategy for the extensiveness of school interference using the 12 core items. The youth and caregiver core interference scores were generated by calculating the average item score. The average item score has been used in exemplar school refusal measures developed for youth (e.g., Kearney, 2002). A separate scoring strategy, which is delineated in a forthcoming manuscript, was developed to assess the type and intensity of school refusal leading up to a PHP admission as well as the functions of that behavior (see Rohrig et al., 2021).

Measures used to establish convergent validity

School refusal

The SRAS-R (Kearney, 2002) was included to examine convergent validity of the SIQ. The SRAS-R is a 24-item measure used to identify variables maintaining school refusal behavior. There are child- and parent-report versions of the measure. Items are measured on a 7-point (0–6) Likert-type scale. Items load onto one of the four functions of school refusal: (a) avoidance of school-based stimuli that provoke a general sense of negative affectivity (anxiety and depression), (b) escaping aversive school-based social and/or evaluative situations, (c) pursuit of attention from significant others, and (d) pursuit of tangible reinforcers outside of school. Subscale scores are generated by calculating the average item score for each of the four functions. Both the child- and the parent-report versions of this instrument have demonstrated good reliability and validity (Kearney, 2002).

Functional impairment

The WHODAS 2.0 (Üstün, Chatterji, et al., 2010) was included to examine convergent validity of the SIQ. The WHODAS 2.0 is a 12-item assessment (Likert-type scale 0–4) of overall functioning for health and disability. Scores are calculated as a percentage, with 0% indicating no disability and 100% indicating full disability. The WHODAS 2.0 has demonstrated high internal consistency (α = .86) and can be self-administered or administered by proxy (e.g., completed by a caregiver about the youth; Üstün, Chatterji, et al., 2010).

School avoidance and anxiety

The child- and parent-report versions of the Screen for Child Anxiety Related Emotional Disorders (SCARED-C and SCARED-P, respectively), 41-item self-report measures that are used to screen for signs of anxiety disorders in children (Birmaher et al., 1999), were used to examine construct validity. SCARED items are scored on a 3-point Likert-type scale from 0 (not true or hardly every true) to 3 (very true or often true). The measure yields a composite total anxiety score as well as subscales for panic disorder, generalized anxiety, separation anxiety, social phobia, and school avoidance. The school avoidance subscale of the SCARED was hypothesized to be more strongly related to the SIQ relative to the other anxiety subscales. The SCARED is a reliable and valid instrument for screening childhood anxiety disorders in clinical settings (Birmaher et al., 1999).

Depression

The Patient Health Questionnaire-9 (PHQ-9; Kroenke et al., 2001) is a 9-item brief depression severity self-report measure. While the PHQ-9 does not directly evaluate functional impairment or school-related constructs, it was included in our examination of convergent validity because it is a measure of symptom severity of common mental health problems in youth and due to the dearth of other relevant school-related measures. No caregiver-report version of this measure is available. Items are scored on a 4-point Likert-type scale from 0 (not at all) to 3 (nearly every day). Higher scores reflect higher severity of depression symptoms. The PHQ-9 is widely used in clinical settings with excellent reliability (α = .89) and good validity (Kroenke et al., 2001). This measure was included to examine convergent validity of the SIQ.

Data Analyses

Descriptive statistics were calculated to examine the frequency, mean, standard deviation, and range for all the measures reported on in this article. An average school interference score was calculated by summing the nonmissing Items 1 through 12 from the youth SIQ and dividing by the number of nonmissing values. Spearman correlations were used to examine the relationship between the 12 school interference questions. Cronbach’s alpha was calculated to report internal consistency.

Exploratory factor analyses using principal axis factoring were conducted for the 12 school interference questions to identify any latent construct. Assumptions were tested using Kaiser–Meyer–Olkin Measure of Sampling Adequacy and Bartlett’s Test of Sphericity. A parallel analysis engine was used to compare eigenvalues from our data set with eigenvalues calculated from randomly generated correlation matrices (Hayton et al., 2004; Patil et al., 2007). Varimax rotation was used to examine factor loadings. The average school interference score, SCARED-C (total and subscales), PHQ-9, SRAS-R, and WHODAS 2.0 were examined for construct validity using Pearson correlations.

To assess divergent validity, a new variable was created. The sample was divided into quartiles based on time of study enrollment. Divergent validity was assessed by comparing the strength of the relationship between the SIQ and time of enrollment relative to the strength of the relationships between the SIQ and other measures of symptoms and functioning using Pearson correlations. The average caregiver school interference score was also calculated using Items 1 through 12 from the caregiver SIQ. Spearman correlations were used to examine the correlation between the average youth school interference score and the average caregiver school interference score, and a paired samples t test was conducted to determine whether youth average interference scores differed significantly from caregiver average interference scores. All analyses were conducted using SPSS version 18.

Handling missing data

Participants (youth) were excluded if they were missing more than 50% (i.e., 17 or more questions) of the items in the SIQ or if they were missing six or more of the interference questions (Items 1–12). Similarly, caregivers were excluded if they were missing six or more of the interference questions (Items 1–12). Youth were included regardless of caregiver completion. In this study, six youth and two caregivers were not included due to missing data according to the criteria above.

For the SCARED-C/P total score, we followed the developer’s recommendations for handling missing data. One youth participant and three caregivers were excluded due to missing more than 20% of items in the measure. In reporting SCARED-C/P subscales, cases were also excluded if they were missing more than 20% of the items in each subscale. No participants were excluded when calculating the PHQ-9 score. SRAS-R function scores were calculated by dividing the sum by the number of items in the function.

Following Heyne et al. (2017), participants were excluded if they were missing two or more items within each function of the SRAS-R: Function 3 (n = 1), Function 4 (n = 2). The same exclusions were used for caregivers completing the SRAS-R: Function 1 (n = 2), Function 2 (n = 4), Function 3 (n = 5), Function 4 (n = 5). For the WHODAS, two participants and two caregivers were excluded due to missing more than one item. Following procedures used by Üstün, Kostanjsek, et al. (2010), missing items for two participants were replaced by the series mean of the other items if only one item was missing.

Results

Table 3 lists the descriptive statistics for all measures for the study with the exception of the SIQ. Table 2 presents SIQ item frequencies and descriptive statistics. No sex differences or differences by age were found on the SIQ.

Table 3.

Measure Descriptives.

Measure	Youth report				Caregiver report
Measure	n	M	SD	Range	n	M	SD	Range
SCARED	103	36.35	17.32	0–78	93	28.71	15.64	0–70
SCARED subscales
Panic Disorder	103	9.98	6.96	0–26	93	6.92	5.84	0–26
GAD	102	11.84	5.06	0–18	92	9.39	5.09	0–18
Separation Anxiety	103	4.00	3.14	0–14	90	2.84	2.83	0–13
Social Phobia	103	7.04	4.49	0–14	93	6.20	4.47	0–14
School Avoidance	96	3.78	2.33	0–8	92	3.67	2.42	0–8
PHQ-9	84	13.15	7.15	0–25
SRAS-R Functions
Function 1: Avoidance of negative affect	104	3.25	1.74	0–6	94	3.14	1.74	0–6
Function 2: Escaping aversive social/school situations	104	1.63	1.32	0–5.50	92	1.91	1.58	0–6
Function 3: Pursuit of attention	103	1.78	1.45	0–5.17	91	1.61	1.53	0–5.83
Function 4: Pursuit of tangible rewards	102	2.28	1.10	0–4.67	91	1.85	1.09	0–5.50
WHODAS 2.0	100	0.25	0.17	0–0.82	89	0.30	0.18	0–0.90

Note. Caregivers were not asked to complete the PHQ-9. SCARED = Screen for Child Anxiety Related Emotional Disorders; GAD = Generalized Anxiety Disorder; PHQ-9 = Patient Health Questionnaire-9; SRAS-R = School Refusal Assessment Scale–Revised; WHODAS = World Health Organization Disability Assessment Schedule.

Reliability

Table 4 presents the correlations among the 12 items in the SIQ measure. Reliability analysis on all 12 items of the SIQ revealed excellent internal consistency (Cronbach’s α = .91).

Table 4.

Spearman Correlations of School Interference Questions.

Item	1	2	3	4	5	6	7	8	9	10	11	12
1. . . . goals you have for the future?
2. . . . ability to apply for college?	.46**
3. . . . getting the grades you want?	.54**	.22*
4. . . . concentrating on schoolwork?	.63**	.35**	.73**
5. . . . staying in the classroom?	.45**	.30**	.36**	.36**
6. . . . setting educational goals?	.59**	.41**	.57**	.60**	.57**
7. . . . studying?	.58**	.38**	.62**	.68**	.45**	.62**
8. . . . completing your homework?	.52**	.36**	.73**	.68**	.49**	.58**	.83**
9. . . . completing work in class?	.46**	.26*	.65**	.62**	.49**	.56**	.56**	.72**
10. . . . being enrolled in the school you would like?	.33**	.27*	.37**	0.19	.36**	.28**	.37**	.35**	.32**
11. . . . your scores on standardized tests?	.35**	.41**	.45**	.33**	.42**	.38**	.51**	.50**	.43**	.55**
12. . . . taking a full course load?	.53**	.42**	.64**	.58**	.58**	.64**	.65**	.67**	.53**	.36**	.71**
Average SIQ score	.71**	.56**	.77**	.75**	.66**	.78**	.82**	.83**	.74**	.54**	.71**	.84**

Note. All items begin with “How much have anxiety, depression, or other mental health problems gotten in the way of . . .” SIQ = School Interference Questionnaire.

p < .05. **p < .01.

Factor Structure

The Kaiser–Meyer–Olkin Measure of Sampling Adequacy had a high value of 0.85 and the Bartlett test of sphericity was significant (p < .01) indicating that correlations were sufficiently large for exploratory factor analysis. In the factor analysis, two factors were initially identified for the 12 school interference items, as they had eigenvalues greater than one (Factor 1 = 6.43, Factor 2 = 1.29). The scree plot (see Figure 1) and parallel analysis resulted in keeping one factor. Reliability analysis confirmed that Cronbach’s alpha was not significantly higher by deleting any items (Cronbach’s α = .90–.92). Results suggested that the Cronbach’s alpha was similar with all 12 items included. Ultimately, we determined one factor was appropriate for the school interference questions. Figure 1 presents the scree plot indicating that one component would be the best model fit for the 12 items in the SIQ.

Figure 1.

Scree plot for School Interference Questionnaire.

Construct Validity

In evaluating the correlations between the SIQ and other measures (see Table 5), the SIQ demonstrated moderate, positive correlations with the SCARED-C, PHQ-9, and WHODAS. Among the SCARED-C subscales, the SIQ had the strongest correlation with the school avoidance subscale (r = .584**), with similar but slightly weaker correlations with the Generalized Anxiety Disorder (GAD) subscale (r = .582**) and the overall SCARED-C score (r = .581**). Among the correlations between the SIQ and four functions of the SRAS, the SIQ exhibited the strongest relationship with Function 1: avoidance of school-based stimuli that provoke a general sense of negative affectivity (r = .61**), and the weakest relationship with Function 4: pursuit of tangible reinforcers outside of school (r = −.05). In review of the correlations with divergent constructs (see Table 5), the SIQ exhibited a weak relationship with the time of study enrollment variable. These results provide initial convergent validity that the SIQ score is more strongly correlated to theoretically related constructs. A moderate, positive correlation (r = .36**) was seen when examining the correlations between the average youth school interference score and the average caregiver interference score. Caregiver SIQ scores were significantly higher (M = 2.57, SE = 0.11) than youth SIQ scores (M = 2.31, SE = 0.51), t(95) = −2.03, p = .05.

Table 5.

Pearson Correlation Matrix Among Average SIQ and Other Measures of Symptomatology and Functioning.

Measure	Average SIQ score	SCARED-C	Panic Disorder	GAD	Separation Anxiety	Social Phobia	School Avoidance	PHQ-9	SRAS Function 1	SRAS Function 2	SRAS Function 3	SRAS Function 4	WHODAS 2.0
Average SIQ score
SCARED-C	.58**
Panic Disorder	.43**	.86**
GAD	.58**	.84**	.62**
Separation Anxiety	.39**	.73**	.55**	.52**
Social Phobia	.32**	.69**	.38**	.47**	.47**
School Avoidance	.58**	.65**	.51**	.49**	.35**	.31**
PHQ-9	.57**	.71**	.59**	.66**	.41**	.47**	.52**
SRAS Function 1	.61**	.57**	.38**	.54**	.31**	.36**	.71**	.49**
SRAS Function 2	.41**	.50**	.30**	.42**	.28**	.49**	.52**	.34**	.66**
SRAS Function 3	.40**	.52**	.30**	.37**	.54**	.46**	.52**	.25*	.45**	.55**
SRAS Function 4	−.05	−.20*	−.12	−.15	−.10	−.28**	−.11	−.12	−.07	−.1	−.03
WHODAS 2.0	.57**	.63**	.55**	.48**	.51**	.43**	.45**	.66**	.38**	.37**	.29**	−.07
Time of enrollment	−.02	−.07	−.05	−.02	−.07	−.06	−.04	.09	.04	−.02	−.05	.08	.05

Note. SIQ = School Interference Questionnaire; SCARED-C = Screen for Child Anxiety Related Emotional Disorders, Child Version; GAD = Generalized Anxiety Disorder; PHQ-9 = Patient Health Questionnaire-9; SRAS-R = School Refusal Assessment Scale–Revised; Function 1 = avoidance of school-based stimuli that provoke a general sense of negative affectivity; Function 2 = escaping aversive school-based social and/or evaluative situations; Function 3 = pursuit of attention; Function 4 = pursuit of tangible reinforcers outside of school; WHODAS = World Health Organization Disability Assessment Schedule.

p < .05. **p < .01.

Discussion

We aimed to expand the knowledge base about school-related impairment by developing a measure that assesses academic functional impairment related to mental health problems in youth along with the frequency, type, and intensity of missed school. We hope this measure will add to the available tools for understanding academic functional impairment and school refusal in clinical settings and for research purposes, and we hope that this measure is particularly helpful to assess academic functional impairment in the midst of stressors related to the COVID-19 pandemic.

The initial psychometric properties of the measure are good. The core interference items exhibited excellent reliability in our sample. Although we attempted to assess interference across multiple domains of school, results were suggestive of a single factor for the school interference questions. As a result, we found that calculating a single average school interference item score was an appropriate scoring strategy for our 12 items.

The pattern of correlations between the SIQ and other measures in this study provides support for construct validity. In line with expectations, the magnitude of the correlations between the SIQ and measures of anxiety symptoms, depression symptoms, and overall disability were all in the moderate range from .57 to .58. In addition, the SIQ had the strongest positive relationship with the school avoidance subscale (r = .58) of the SCARED relative to the other subscales, which ranged from .32 to .58. As anticipated, the SIQ also exhibited no relationship with timing of enrollment. There was a moderate, positive correlation between average youth interference scores and average caregiver interference scores, which is consistent with other studies of multi-informant assessments of youth mental health (De Los Reyes et al., 2015, 2019). On average, caregiver SIQ scores were significantly higher than youth scores. Caregivers may have observed more academic functional impairment than youth because parents may bring more of a “bird’s eye view,” with greater awareness of the different ways in which mental health can affect school and academics. These findings underscore that the SIQ appears to be capturing a construct that behaves similarly to what we would theoretically expect from a measure of school interference. Thus, the SIQ can add to the lean but growing list of measures that assess school refusal and school functioning.

The SIQ adds broad measure of academic functional impairment to the literature on school refusal and functional impairment related to mental health problems in youth. This measure is both more comprehensive in its assessment of functional impairment related to school than other available measures of functional impairment, and it provides broader information than available questionnaires related to school refusal. In addition to information about the degree and areas of academic functional impairment, the SIQ provides detailed information about the pattern and intensity of missed school. We hope that the SIQ aids in research related to academic functional impairment and school refusal, and we hope that the SIQ is a useful assessment tool that informs clinical intervention for youth with academic functional impairment due to mental health problems.

Our study also has its limitations. The generalizability of our results is limited because our sample was recruited from a PHP, representing a population that is more severely impaired than outpatient or nonclinical samples. It is therefore not clear that results will generalize outside of acute care settings. Our results may also have been affected by factors related to the families for whom consent was obtained, given that just less than 55% of those admitted to the PHP during the study consented to participate. It is possible that families who were discharged before consent was obtained or who opted not to participate in this study would have responded to the items on this questionnaire in a systematically different manner compared with those who consented. Our study was also limited by the number and type of measures we used. For example, we used other self-report measures to establish convergent validity and did not use other types of data, such as school attendance records, school avoidance behavior on the PHP unit, or grades. In addition, our results excluded adolescents admitted to the PHP during school breaks, which might have affected our results.

Future research should examine the use of the SIQ in other populations, including those receiving outpatient mental services. Given that the majority of caregiver participants in our study held postgraduate degrees, and the majority of youth identified as Caucasian and/or non-Latino, future research should also examine the SIQ in more ethnically diverse families and in families with caregivers from more diverse educational backgrounds. An important next step will be examining other elements of the SIQ (e.g., items that assess the type and intensity of missed school in the past week and month, items assessing how youth spend their time at home) for both the youth-report and the caregiver-report versions of the questionnaire. Exploring the concurrent validity of the self-report SIQ measure with other measures of school refusal, such as formal attendance records, will provide additional data on the utility of the questionnaire. Another direction for future research is developing a school staff-report version of the SIQ, examining the psychometric properties of this measure, and evaluating agreement between youth, caregiver, and school staff versions of this measure.

This study provides initial support for the reliability and construct validity of a novel measure of academic functional impairment. Our findings suggest that the core interference items on the SIQ are represented by a single factor. Further study of the SIQ in outpatient and more diverse populations will expand our knowledge about academic functional impairment.

Footnotes

Acknowledgements

We would like to thank the staff at the NewYork-Presbyterian/Weill Cornell Medicine Adolescent Partial Hospitalization Program for supporting this work. A huge thank you also goes to the families who participated in this research and to the generous donors who support the Youth Anxiety Center.

Declaration of Conflicting Interests

Shannon Bennett has received grant funding from the Tourette Association of America. She is an unpaid member of the Medical Advisory Board of the Tourette Association of America and has received honoraria from the Tourette Association of America for speaking engagements and as a trainer in the Tourette Syndrome Behavior Therapy Institute. Dr. Bennett also receives royalties from Wolters Kluwer for an entry for UpToDate on the topic of pediatric anxiety.

Funding

This work is supported by the New York Presbyterian Youth Anxiety Center.

ORCID iD

Elaina A. Zendegui

References

American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). American Psychiatric Publishing.

Andrews

Kemp

Sunderland

Von Korff

Üstün

T. B.

(2009). Normative data for the 12 item WHO Disability Assessment Schedule 2.0. PLOS ONE, 4(12), Article e8343.

Becker

K. D.

Brandt

N. E.

Stephan

S. H.

Chorpita

B. F.

(2014). A review of educational outcomes in the children’s mental health treatment literature. Advances in School Mental Health Promotion, 7(1), 5–23.

Birmaher

Brent

D. A.

Chiappetta

Bridge

Monga

Baugher

(1999). Psychometric properties of the Screen for Child Anxiety Related Emotional Disorders (SCARED): A replication study. Journal of the American Academy of Child & Adolescent Psychiatry, 38(10), 1230–1236.

De Los Reyes

Augenstein

T. M.

Wang

Thomas

S. A.

Drabick

D. A. G.

Burgers

D. E.

Rabinowitz

. (2015). The validity of the multi-informant approach to assessing child and adolescent mental health. Psychological Bulletin, 141, 858–900.

De Los Reyes

Makol

B. A.

Racz

S. J.

Youngstrom

E. A.

Lerner

M. D.

Keeley

L. M

. (2019). The Work and Social Adjustment Scale for Youth: A measure for assessing youth psychosocial impairment regardless of mental health status. Journal of Child and Family Studies, 28, 1–16.

Epstein

M. H.

(1999). The development and validation of a scale to assess the emotional and behavioral strengths of children and adolescents. Remedial and Special Education, 20(5), 258–262.

Fabiano

G. A.

Pelham

W. E.

(2016). Impairment in children. In Goldstein

Naglieri

J. A.

(Eds.), Assessing impairment: From theory to practice (pp. 71–89). Springer.

Gadow

K. D.

Kaat

A. J.

Lecavalier

(2013). Relation of symptom-induced impairment with other illness parameters in clinic-referred youth. Journal of Child Psychology and Psychiatry, 54(11), 1198–1207.

10.

Goldstein

Naglieri

J. A.

(2016). Assessing impairment: From theory to practice. Springer.

11.

Hayton

J. C.

Allen

D. G.

Scarpello

(2004). Factor retention decisions in exploratory factor analysis: A tutorial on parallel analysis. Organizational Research Methods, 7(2), 191–205.

12.

Heyne

D. A.

Vreeke

L. J.

Maric

Boelens

Van Widenfelt

B. M.

(2017). Functional assessment of school attendance problems: An adapted version of the School Refusal Assessment Scale-Revised. Journal of Emotional and Behavioral Disorders, 25(3), 178–192.

13.

Hoagwood

Jensen

P. S.

Petti

Burns

B. J.

(1996). Outcomes of mental health care for children and adolescents: I. A comprehensive conceptual model. Journal of the American Academy of Child & Adolescent Psychiatry, 35(8), 1055–1063.

14.

John

Gammon

D. G.

Prusoff

B. A.

Warner

(1987). The Social Adjustment Inventory for Children and Adolescents (SAICA): Testing of a new semistructured interview. Journal of the American Academy of Child & Adolescent Psychiatry, 26(6), 898–911.

15.

Kearney

C. A.

(2002). Identifying the function of school refusal behavior: A revision of the School Refusal Assessment Scale. Journal of Psychopathology and Behavioral Assessment, 24(4), 235–245.

16.

Kearney

C. A.

(2003). Bridging the gap among professionals who address youths with school absenteeism: Overview and suggestions for consensus. Professional Psychology: Research and Practice, 34(1), 57–65.

17.

Kearney

C. A.

(2006). Confirmatory factor analysis of the School Refusal Assessment Scale-Revised: Child and parent versions. Journal of Psychopathology and Behavioral Assessment, 28(3), 139–144.

18.

Kearney

C. A.

(2008). School absenteeism and school refusal behavior in youth: A contemporary review. Clinical Psychology Review, 28(3), 451–471.

19.

Kearney

C. A.

Albano

A. M.

(2004). The functional profiles of school refusal behavior: Diagnostic aspects. Behavior Modification, 28(1), 147–161.

20.

Kearney

C. A.

Silverman

W. K.

(1993). Measuring the function of school refusal behavior: The School Refusal Assessment Scale. Journal of Clinical Child Psychology, 22(1), 85–96.

21.

Kearney

C. A.

Silverman

W. K.

(1996). The evolution and reconciliation of taxonomic strategies for school refusal behavior. Clinical Psychology: Science and Practice, 3(4), 339–354.

22.

Kennair

Mellor

Brann

(2011). Evaluating the outcomes of adolescent day programs in an Australian child and adolescent mental health service. Clinical Child Psychology and Psychiatry, 16(1), 21–31.

23.

Knollmann

Knoll

Reissner

Metzelaars

Hebebrand

(2010). School avoidance from the point of view of child and adolescent psychiatry. Deutsches Ärzteblatt International, 107(4), 43–49.

24.

Kroenke

Spitzer

R. L.

Williams

J. B.

(2001). The PHQ-9: Validity of a brief depression severity measure. Journal of General Internal Medicine, 16(9), 606–613.

25.

Langley

A. K.

Bergman

R. L.

McCracken

Piacentini

J. C.

(2004). Impairment in childhood anxiety disorders: Preliminary examination of the Child Anxiety Impact Scale–Parent Version. Journal of Child and Adolescent Psychopharmacology, 14(1), 105–114.

26.

Langley

A. K.

Falk

Peris

Wiley

J. F.

Kendall

P. C.

Ginsburg

Birmaher

March

Albano

A. M.

Piacentini

(2014). The Child Anxiety Impact Scale: Examining parent-and child-reported impairment in child anxiety disorders. Journal of Clinical Child & Adolescent Psychology, 43(4), 579–591.

27.

McDermott

B. M.

McKelvey

Roberts

Davies

(2002). Severity of children’s psychopathology and impairment and its relationship to treatment setting. Psychiatric Services, 53(1), 57–62.

28.

Park

J. H.

Lee

S. I.

Schachar

R. J.

(2011). Reliability and validity of the Child and Adolescent Functioning Impairment Scale in children with Attention-Deficit/Hyperactivity Disorder. Psychiatry Investigation, 8(2), 113–122.

29.

Patil

V. H.

Singh

S. N.

Mishra

Donovan

(2007). Parallel analysis engine to aid determining number of factors to retain [Computer software]. https://analytics.gonzaga.edu/parallelengine/

30.

Rapee

R. M.

(2012). Family factors in the development and management of anxiety disorders. Clinical Child and Family Psychology Review, 15(1), 69–80.

31.

Rehm

Üstün

T. B.

Saxena

Nelson

C. B.

Chatterji

(1999). On the development and psychometric testing of the WHO screening instrument to assess disablement in the general population. International Journal of Methods in Psychiatric Research, 8(2), 110–122.

32.

Rohrig

Bennett

S. M.

Desai

Zendegui

E. A.

Chiu

A. W.

(2021). A description of school refusal behavior in adolescents prior to acute care admission [Manuscript in preparation]. Department of Psychiatry, Weill Cornell Medicine.

33.

Rohrig

Desai

Zendegui

Bennett

Chiu

(2020, November). School refusal in adolescents prior to acute care admission [Poster presentation]. 54th annual Convention for the Association for Behavioral and Cognitive Therapies, Philadelphia, PA, United States.

34.

Sato

A. F.

Hainsworth

K. R.

Anderson Khan

Ladwig

R. J.

Weisman

S. J.

Hobart Davies

(2007). School absenteeism in pediatric chronic pain: Identifying lessons learned from the general school absenteeism literature. Children’s Healthcare, 36(4), 355–372.

35.

Sawyer

Arney

Baghurst

Clark

Graetz

Kosky

Nurcombe

Patton

G. C.

Prior

M. R.

Raphael

Rey

Whaites

L. C.

Zubrick

S. R.

(2000). The mental health of young people in Australia: Child and adolescent component of the National Survey of Mental Health and Wellbeing. Department of Health and Aged Care, Mental Health and Special Programs Branch.

36.

Thatte

Makinen

J. A.

Nguyen

H. N.

Hill

E. M.

Flament

M. F.

(2013). Partial hospitalization for youth with psychiatric disorders: Treatment outcomes and 3-month follow-up. The Journal of Nervous and Mental Disease, 201(5), 429–434.

37.

Üstün

T. B.

Chatterji

Kostanjsek

Rehm

Kennedy

Epping-Jordan

Saxena

von Korff

Pull

(2010). Developing the World Health Organization disability assessment schedule 2.0. Bulletin of the World Health Organization, 88, 815–823.

38.

Üstün

T. B.

Kostanjsek

Chatterji

Rehm

(2010). Measuring health and disability: Manual for WHO disability assessment schedule WHODAS 2.0. World Health Organization.

39.

Winters

N. C.

Collett

B. R.

Myers

K. M.

(2005). Ten-year review of rating scales, VII: Scales assessing functional impairment. Journal of the American Academy of Child & Adolescent Psychiatry, 44(4), 309–338.