Profiles of Third-Grade Emotional,Behavioral,and Academic Indicators: Co-Occurrence and Distal Outcomes in Sixth Grade

Abstract

The purpose of this study was to identify emotional, behavioral, and academic co-occurring risk profiles of third-grade youth and to understand how they are longitudinally associated with later outcomes in sixth grade. The latent profile analysis of data from a U.S. Midwest community sample of 1,785 third graders resulted in five classes of risk profiles: No Risk (47%), Low Risk (36%), High Internalizing and Emotion Dysregulation (8%), High Externalizing (7%), and Co-occurring Risk (2%). The Co-occurring class had significantly worse outcomes on all sixth-grade measures compared to other classes. The High Internalizing and Emotion Dysregulation class had significantly higher internalizing problems but also higher externalizing problems. Youth with predominant externalizing features continued to evidence these problems in sixth grade, along with a high number of office discipline referrals, in-school suspensions, and out-school suspensions. Students with co-occurring social, emotional, behavioral, and academic issues are at increased risk of future challenges. Schools can use comprehensive and systematic screening to identify students with co-occurring problems. The study highlights the importance of early identification through screening and taking into consideration the co-occurring presentation of symptoms.

Keywords

prevention co-occurring risk screening elementary school middle school

Emotional, behavioral, and academic problems commonly co-occur. Youth who experience co-occurring emotional, behavioral, and academic challenges face a unique set of hurdles that can significantly impact their long-term success. Although singular emotional, behavioral, and academic problems are related to negative consequences for youth, their co-occurrence amplifies the harmful effects both in the short and long term (Darney et al., 2013; Reinke et al., 2008; Shi & Ettekal, 2021). These intertwined difficulties often lead to negative outcomes such as school dropout, involvement in the juvenile justice system, and mental health disorders (Darney et al., 2013; Reinke et al., 2008). The following provides a summary of research on the consequences of co-occurring problems and a conceptual framework for understanding co-occurrence and why it can have such deleterious outcomes for youth.

Outcomes Associated With Co-Occurring Problems

Several longitudinal studies have investigated the impact of co-occurring problems among youth using sophisticated methodologies—like latent class (LCA) or profile analysis (LPA) and growth mixture modeling (GMM)—to simultaneously model co-occurring patterns. In one study with predominately Black youth (N = 678), Reinke and colleagues (2008) used LCA and found that, regardless of gender, youth with co-occurring academic and behavioral difficulties at school entry were more likely to have unfavorable outcomes in sixth grade than youth who have no academic or behavior risk (no risk), as well as those with only academic or only behavior problems. Compared to boys in the No Risk class, boys in the co-occurring academic and behavior problem class had 11 times the chance of being placed in special education, 11 times the chance of being classified as having conduct problems by teachers, and 7 times the chance of being suspended from school. In the case of girls, both the co-occurring academic and behavior class and the academic problem–only class had three times the likelihood of getting poor grades compared to girls in the No Risk class. The co-occurring problem class for girls had a greater likelihood of school suspension, conduct problems, and deviant peer relationships in sixth grade. Darney and colleagues (2013) examined outcomes from this same set of youth through twelfth grade. Results revealed that both boys and girls with co-occurring academic and behavior difficulties in first grade had a higher chance of having poor math and reading scores, to be receiving special education services, not graduating from high school, and receiving supports for mental health problems in twelfth grade, showing that these problems were persistent throughout the developmental course.

Using LPA, C. C. Chen and colleagues (2020) examined the adjustment of approximately 3,000 sixth graders, including 415 students with disabilities, during their transition to middle school. Teacher ratings of externalizing, internalizing, and relational qualities were utilized to identify several distinct interpersonal competence patterns. Findings indicated that students with disabilities were overrepresented in poor adjustment profiles and that students in these poor adjustment profiles were more likely to report having difficulty transitioning to middle school and that school was a negative context for them (C. C. Chen et al., 2020).

In another study, Shi and Ettekal (2021) examined patterns of co-developing internalizing and externalizing problems from early childhood through adolescence (i.e., Grades 1–12) using parallel-process GMM. They identified subgroups of children with heterogeneous developmental trajectories (i.e., pure and co-occurring internalizing and externalizing problems) and examined their long-term associations with teacher–child relationship quality and academic (math and reading) performance (Shi & Ettekal, 2021). Results revealed four distinct trajectories of internalizing and externalizing problems including chronic co-occurring, moderate co-occurring, pure-externalizing, and low-risk groups. Children with chronic co-occurring internalizing and externalizing problems exhibited more sustained teacher-child conflict, lower teacher-child warmth, and lower math and reading performance. Compared to children in the low-risk group, those in all three risk groups exhibited patterns of academic problems that were either sustained or worsened after the transition to middle school.

A study by C. C. Chen et al. (2022) studied how problem behaviors (internalizing and externalizing) and academic performance affect each other from kindergarten to third grade. Findings showed a reciprocal relationship where higher problem behaviors predicted poor academic performance, and lower academic performance increased problem behaviors which underscores the importance to understand these problems early on to address these interconnected issues. Furthermore, a recent longitudinal study using data from the Study of Early Child Care and Youth Development (N = 1,048) examined the cascading and reciprocal effect of internalizing problems on future school achievement. Results revealed that internalizing problems in elementary school for girls were related to their school achievement in high school and that internalizing problems for boys in ninth grade had a concurrent association with their cognitive achievement (Okano et al., 2020).

Developmental Cascades Model

The developmental cascades model is a prominent theory that explains both the reason for co-occurring problems and their compounding effects (Patterson, et al., 1992). According to this model, some children enter school with problematic social interaction patterns that are learned and reinforced from parenting practices at home. Patterson referred to these patterns as coercive interactions and described the process whereby adults and children negatively reinforce the escalation of hostile and aggressive interactions with each other (see also S. A. Burt et al., 2021). Children with pre-existing internal risk characteristics such as inattention, emotional dysregulation, and expressive language deficits are more vulnerable to these patterns of interaction (Carpenter & Drabick, 2011; Chow & Wehby, 2018; Moffitt, 1993; Yang et al., 2022). Children are likely to repeat these entrenched patterns with new adults and peers they encounter at school (See Reinke & Herman, 2002). Coercive and aversive interactions with educators and peers increase the likelihood that children experience social rejection and exclusion from the classroom setting. Social and academic failure experiences, dual failures, in turn contribute to negative self-perception, deviant peer affiliation, and lagging social skills. Over the elementary years, these experiences cumulate and increase children’s risk of internalizing symptoms, externalizing behaviors, and school disengagement and failure.

Previous studies confirmed the cascading pathways from externalizing problems to subsequent academic performance and from academic performance to internalizing problems (K. B. Burt & Roisman, 2010; Masten et al., 2005; Moilanen et al., 2010; van Lier et al., 2012). Research suggests that externalizing and internalizing symptoms not only predict subsequent academic difficulties but also increase vulnerability to further behavioral challenges across domains (Moilanen et al., 2010; Shi & Ettekal, 2021; Van der Ende et al., 2016). Children with chronic co-occurring internalizing and externalizing problems showed the most persistent deficits in academic achievement from Grade 1 through Grade 9, even after controlling for various demographic factors (Shi & Ettekal, 2021). The results of Shi and Ettekal’s (2021) study align with previous research (Moilanen et al., 2010; Vaillancourt et al., 2013; Zimmermann et al., 2013), highlighting the persistent negative impact of co-occurring internalizing and externalizing problems on academic performance. Research also supports the notion of a bidirectional, transactional relationship between behavioral problems and academic achievement, where each influences the other over time (Metsäpelto et al., 2020; Zimmermann et al., 2013).

Further evidence of the cascade model comes from the robust literature on homotypic and heterotypic development of psychopathology (Cicchetti & Toth, 1998; Costello et al., 2011; Moffitt, 1993). Homotypic development of psychopathology refers to the persistence of the same type of symptoms or disorder over time (e.g., anxiety in childhood continuing as anxiety in adulthood). Heterotypic development describes the progression of one type of symptoms or disorder into a different type over time (e.g., childhood inattention evolving into depression in adulthood; Cicchetti & Toth, 1998). As an example, using a community sample of elementary age children, Reinke and Ostrander (2008) found evidence of homotypic and heterotypic continuity in baseline characteristics and their symptoms 5 years later. Notably, inattention and conduct problems were stable over time regardless of age or gender. In addition, inattention predicted subsequent depression, and depression in young children was a unique risk factor for subsequent conduct problems.

In addition, internalizing and externalizing problems often share precursors such as low self-regulation, interpersonal conflicts, and genetic or biological influences, indicating a susceptibility to co-occurrence of these challenges (Cosgrove et al., 2011; Woltering & Shi, 2016). The development of children’s self-regulatory abilities, in particular, is likely associated with differences in developmental trajectories of externalizing behavior problems (Calkins & Perry, 2016). Research suggests that greater self-regulatory skills, including emotion regulation and inhibitory control, are associated with a lower and more stable trajectory of externalizing behaviors. Specifically, higher levels of emotion regulation, as reported by both parents and through observations, increase the likelihood of children following a low/stable externalizing trajectory, while these skills also predict a decreasing trajectory of externalizing behaviors over time (Perry et al., 2018).

Given the interplay between externalizing and internalizing problems, and that self-regulation, inattention, and academic competence are often antecedents or consequences of these problems, understanding how the constellation of these problems early in development are associated with later outcomes is important. Knowing whether particular profiles of problems early in schooling lend themselves to challenges across development can help guide prevention efforts and supports to youth. Given the literature reviewing the common co-occurrence of these four problem areas (externalizing, internalizing, emotional dysregulation, and attention/academic concerns), it is important not only to understand the risks associated with each in isolation but also to examine their overlapping patterns of impact on youth at a later age. Understanding the impact of the co-occurrence of problem areas can lead to needed interventions early to prevent later more entrenched challenges to academic and behavioral health.

Universal Screening as a Source of Data

Many schools have adopted a multi-tiered system of support (MTSS; McIntosh & Goodman, 2016). A key feature of a well-functioning tiered system is the systematic use of universal screening (see Herman et al., 2023). Universal screening assists school decision-makers in efficiently identifying at-risk students and placing them in appropriate interventions. Importantly, if the universal screener has multiple subscales and is comprehensive in nature, the data can provide information about student profiles, allowing for early identification of students with co-occurring problems.

A comprehensive and multi-domain universal screener for understanding patterns of youth co-occurring problems is the Early Identification System (EIS; Herman et al., 2021; Reinke et al., 2022; Thompson et al., 2021). The EIS was developed to align with the developmental cascades model. Items and constructs were selected to tap risk in each of the prominent domains identified by the cascades model including externalizing behaviors, internalizing behaviors, inattention and academic problem, and emotion dysregulation (see Reinke et al., 2022). In addition, the EIS was developed to overcome the usability barriers associated with other universal screeners including their cost and burden. A recent paper provided detailed evidence showing the EIS is feasible and usable by schools over time and with favorable treatment utility and social consequences (Herman et al., 2023).

Youth-Reported Adjustment

Prior studies have examined co-occurring academic and behavior problems at school entry as risk for future maladjustment (Darney et al., 2013; Reinke et al., 2008). Such studies necessarily have relied on adult perspectives given the young age of youth and challenges of self-report. By the time youth are 8 or 9 years old (third grade), they are developing the cognitive capacity to provide accurate self-reports and are developing more stable cognitions about themselves and their relationships (Riley, 2004). Notably, studies have consistently found youth in third grade and beyond can accurately report on their academic, behavior, and emotional functioning, and these reports relate in predictable ways to external criteria such as academic performance, discipline referrals, and attendance (Herman et al., 2021; Thompson et al., 2021; Reinke et al., 2022). Including youth reports of emotional distress is especially important given that self-report is the gold standard for identifying internalizing symptoms, and adult informants often miss these symptoms in children (Herman et al., 2018). Thus, capturing youth perspectives about their own adjustment during third grade can provide a new lens to consider co-occurring problems and subsequent risks related to them.

Current Study

The purpose of this study was threefold. First, we wanted to identify the number and types of emotional, behavioral, and academic risk profiles for a large sample of third-grade youth providing self-reported data. We hypothesized that four or more profiles could emerge. We expected that there would be one group of youth who demonstrated no risk, a second group that would have predominantly internalizing problems, a third group with predominantly externalizing problems, and a fourth group with co-occurring internalizing and externalizing problems. Finally, we wanted to determine if third-grade profiles were predictive of social, behavioral, and academic indicators in sixth grade. We hypothesized that the higher risk profiles would predict peer relationship problems, attention and academic problems, emotion dysregulation, as well as lower attendance, higher number office discipline referrals, and in- and out-of-school suspension, as well as self-reported internalizing and externalizing problems. While schools implementing MTSS may use screening to identify and intervene, the interventions are often designed to target a specific problem area rather using profiles of data to determine the type and level of intervention. As such, understanding how third-grade profiles are longitudinally associated with later outcomes in sixth grade can be helpful. Knowing that risk profiles are linked to persistent behavioral and academic risk can inform how schools use screening data to intervene in a more comprehensive manner in efforts to prevent these later issues going into middle school.

Methods

Participants

The participating third-grade youth (N = 1,785) were situated in six school districts from the U.S. Midwest participating in a county-wide school mental health program that included universal screening, conducted three times per year. These existing data were completely de-identified when available for use in this study. Five of the six school districts are considered rural, and the sixth and parochial schools were in a suburban setting. Of the youth participants, 53% were male, 67.6% white, 16.3% Black, 7.5% Multiracial, 3.6% Latine, 4.1% Asian, 0.4% American Indian, and 0.1% were Pacific Islander. Forty-three percent of the sample received free or reduced meals (FRM), which is a proxy for social economic status (SES).

Measures

Early Identification System-Student Report

The Early Identification System-Student Report (EIS-SR) is a universal social, emotional, and behavioral screening instrument administered electronically to students that has seven subscales, including attention and academic problems, peer relationship problems, internalizing problems, externalizing problems, emotion dysregulation, school disengagement, and bullying behaviors. An initial investigation with 1,590 youth elementary and middle schools supported the factor structure and psychometric properties of the EIS with excellent model fit and strong scale reliabilities (Huang et al., 2019). Subsequent studies replicated these findings across each developmental level with approximately 5,000 youth in each study (Herman et al., 2021; Reinke et al., 2022; Thompson et al., 2021). The EIS is also sensitive to change (Reinke et al., 2022).

Youth completed the measure three times across the school years (October, January, and April). The EIS-SR was administered in class online to students by teachers or school counselors reading the items using a script. The EIS-SR is written at a first-grade reading level. Youth answered a total of 37 questions, such as “I have trouble finishing my work” and “In the past month I have felt sad.” The survey took youth between 5 and 15 minutes to complete. Response options were Likert-type scales (0 = Never, 1 = Sometimes, 2 = Often, 3 = Always). For the purpose of this study, the following subscales were used to form student risk profiles in third grade: internalizing problems, externalizing problems, emotion dysregulation, and attention and academic problems. The omega reliability coefficient for the internalizing problems subscale was .817. The omega reliability coefficient for the externalizing problems scale was .763. The omega reliability coefficient for the emotion dysregulation scale was .821, and the attention and academic problems scale omega coefficient was .674. Each subscale was converted to a z-score for the third-grade sample prior to analyses. Students with a z-score of 2 standard deviations above their peers for each subscale are considered having high risk in that area, and those who are 1 standard deviation above their peers are considered to have some risk (see Reinke et al., 2018; Thompson et al., 2017).

These same youth completed the EIS-SR in sixth grade. To determine if third-grade risk profiles predicted sixth-grade indicators, the following subscales were used: peer relationship problems, internalizing problems, externalizing problems, emotion dysregulation, and attention and academic problems. Each subscale was converted to a z-score and categorized by cutting the scores so that youth who scored 2 standard deviations above the mean were coded as high risk (1) and others as not having risk in that area (0).

Attendance and School Discipline Data

Schools provided the percent of days attended, the total number of office discipline referrals, the total number of in-school suspensions, and the total number of out-of-school suspensions for each student for sixth grade.

Academic Achievement

Schools provided percentile ranking for the fall administration of the STAR reading and math assessments for all youth. STAR assessments are computer-adaptive tests which adjust the level of difficulty of each question in real time depending on the student’s earlier responses. STAR provides a National Percentile Rank, which compares scores to other youth in the same grade level throughout the county (range 1–99). The STAR reading and math assessments have high internal (.93) and test–retest reliability (.91; Renaissance Learning, 2020).

Demographic Information

Schools provided demographic information for youth, including sex, FRM status, and race.

Analysis

Latent profile analysis was used to examine patterns of student-reported social, behavioral, and academic risk indicators (Nylund et al., 2007; Rosato & Baer, 2012). LPA is a person-centered approach that allowed us to identify “types” of students based on their patterns of risk using the four indicators of co-occurrence (internalizing, externalizing, academic, and emotional regulation). A strength of LPA is that the decisions about type and patterns are model-based and latent such that each student has a given probability of being in each pattern type. Thus, LPA allows researchers to account for error in membership classification and validate the final class solutions against external criteria within the same analyses. In LPA, behaviors are locally independent within each class. For this study, this means that student risk can be explained by an underlying classification of youth into subclasses with similar patterns of behavior. Overall, the goal of LPA is to identify the smallest number of classes that accurately describes the association between the student risk indicators. The results for the characteristics of identified latent profiles were expressed in z-scored levels of student-reported internalizing problems, externalizing problems, emotion dysregulation, and attention and academic problems along with the prevalence or proportion of youth in each class.

All analyses were conducted using MPlus 8.6 (Muthén & Muthén, 1998–2021). In LPA, a combination of statistical considerations and substantive theory is used to decide on the best-fitting model. To determine the relative fit of the models, we compared models with differing numbers of classes using the Akaike information criterion (AIC; Bozdogan, 1987), the Bayesian information criterion (BIC; Schwartz, 1978), and the sample-size-adjusted Bayesian information criterion (aBIC; Sclove, 1987). In these analyses, more weight was given to the BIC (Schwartz, 1978) because simulation studies suggest that the BIC provides the most reliable indicators of true model fit (Nylund et al., 2005). Typically, the smaller the information criteria, the better the model fit to the data. Furthermore, we used a likelihood difference test, the Vuong-Lo–Mendall–Rubin (VLMR; Lo et al., 2001; Vuong, 1989), which assesses the fit between two nested models that differ by one class and provides a p value that indicates which model fits best. In addition, we evaluated the classification precision as indicated by estimated posterior class probabilities, summarized by the entropy measure (Ramaswamy et al., 1993). In addition, a bootstrapped parametric likelihood ratio test (BLRT) procedure was used to confirm the best-fitting model once other model fit indicators, class prevalence, and interpretability were examined (see McLachlan, 1987; Nylund et al., 2007). Other criteria include examining the entropy value, which is an estimate of classification precision (values closer to 1 indicate better fit), theoretical support, group size (classes containing less than 5% of the sample could indicate a poor model), and interpretability (Ferguson et al., 2020; Nylund, 2007).

Next, logistic regression was used to determine if third-grade profiles predicted the outcomes in sixth grade that were categorized to indicate high risk (2 SDs above the mean) for problems on each scale (peer relationship problems, externalizing problems, internalizing problems, attention and academic problems, emotional dysregulation). Two standard deviations above the mean is a typical cutoff point for clinically significant symptoms on behavioral assessment measures. Finally, the Mplus Auxiliary function (Muthén & Muthén, 2010) was used for all continuous external variables while controlling for student race, gender, and FRM. This method derives profile membership based on the observed risk factor scores and uses the posterior probabilities to compute means for each external variable (attendance, disciplinary actions, reading, and math scores in sixth grade). Differences between these mean scores were then tested for statistical significance.

To accommodate for missing data, Mplus software uses full information maximum likelihood with the assumption that the data are missing at random (Little, 1995), a common approach used within this analysis method (Schafer & Graham, 2002). Overall, 100% of the participants had all data for all variables used for the profile analysis; 83% had data for sixth-grade internalizing problems, attention and academic problems, externalizing problems, emotion dysregulation, and peer relationship problems; 76% had data for attendance data; 77% had data for office discipline referrals and in-school suspensions; 73% had data for out-of-school suspensions; and 63% had reading achievement and math achievement data. Chi-square tests revealed no difference between missing data on gender or FRM, but there were significant differences by race, with white, Black, and Asian students being overrepresented in having missing data in ninth grade (χ² (6, N = 1,785) = 53.83 p < .001). All demographic variables were included as covariates in the models. The minimum covariance coverage recommended for reliable model convergence is 0.10. In this study, coverage exceeded .83 for all analyses, well exceeding the recommended minimum coverage.

Results

Table 1 provides the descriptive statistics for the third-grade and sixth-grade variables used in the study. These include EIS-SR, reading, math, and discipline referral means, standard deviations, and range, as well as attendance percentages for the whole sample.

Table 1.

Descriptive Statistics for Study Variables.

Study variable	M	SD	Range
Fall third grade Internalizing problems	6.29	5.00	0–27
Externalizing problems	2.95	2.94	0–20
Emotion dysregulation	3.45	3.21	0–12
Attention/Academic problems	3.27	2.64	0–15
Fall sixth grade
Internalizing problems	4.59	4.43	0–27
Externalizing problems	2.37	2.35	0–15
Peer relationship problems	2.12	2.12	0–10
Emotion dysregulation	2.93	3.06	0–12
Attention/Academic problems	3.12	2.49	0–15
Reading scores	50.82	29-53	1–99
Math scores	48.97	30.30	1–99
Sixth grade total end of year
% Attendance	95.80	4.44	49–100
# ODRs	0.92	3.23	0–47
# In-school suspensions	0.21	0.87	0–12
# Out-of-school suspensions	0.06	0.31	0–4

Note. Means for fall Early Identification System variables represent the raw score before standardizing. ODRs = office discipline referrals.

LPA of Third-Grade Indicators

Student report of internalizing problems, externalizing problems, emotion dysregulation, and attention and academic problems was utilized to determine the optimal number of profiles of student social-emotional and behavior risk. According to the AIC, BIC, and aBIC, significant improvements in model fit were observed in up to six classes, as is common for large samples. The five-class solution had a unique class in comparison to the four-class solution which demonstrated high internalizing problems. However, inspection of the six-class solution indicated that a new class emerged, which mimicked the five-class solution, but the new class represented very few youth (less than 1% for the new class), providing little additional information and not necessarily in line with theory. Thus, the five-class solution was deemed the best-fitting model for the sample. LPA fit indices for class solutions are summarized in Table 2.

Table 2

Model Fit Indices for 1–6 Class Solutions of Student Risk.

Number of classes	AIC	BIC	aBIC	VLMR LRT	Entropy
1 class solution	20311.83	20355.74	20330.33	—	—
2 class solution	18393.15	18464.52	18423.22	0.00	0.87
3 class solution	17902.95	18001.76	17944.57	0.00	0.85
4 class solution	17632.73	17758.98	17685.91	0.00	0.81
5 class solution	17419.46	17573.16	17484.21	0.00	0.81
6 class solution	17247.57	17428.72	17323.88	0.00	0.84

Note. LC = latent class; AIC = Akaike information criterion; BIC = Bayesian information criterion; aBIC = adjusted Bayesian information criterion; VLMR RT = Vuong-Lo–Mendall–Rubin ratio test. Bold formatting indicates best fit: All entropy ratings indicate acceptable fit. Entropy values close to 1.0 indicate higher classification precision.

Figure 1 summarizes the prevalence and characteristics of the five profiles identified. Class 1 was characterized as the No Risk class (47%; n = 841) given that this group’s z-scores were lower than zero for all indicators. Class 2 was characterized as the Low Risk class given z-scores for indicators were very low (36%; n = 635). Class 3 was characterized as High Internalizing and Emotion Dysregulation (8%; n = 140), with z-scores two standard deviations above average on these indicators. Class 4 was characterized as High Externalizing (7%; n = 128) with a z-score of three standard deviations above average for this indicator. Finally, Class 5 was the smallest group and characterized as Co-occurring Risk (2%; n = 41), with z-scores above average across all indicators, with externalizing problems as very high (see Figure 1).

Figure 1.

Mean Z-Scores for Five-Class Solution of Third-Grade Risk Profiles.

Predicting Sixth-Grade Outcomes

Table 3 provides the sixth-grade means, standard error, and equality tests across all profiles of student risk.

Table 3.

Sixth-Grade Means, Standard Error, and Equality Tests Across Profiles of Student Risk (N = 1,785).

Variable	Class 1: No Risk	Class 2: Low Risk	Class 3: High Internalizing & Dysregulation	Class 4: High Externalizing	Class 5: Co-occurring	Overall test of significance	Significant class comparisons
% Attendance	96.23 (0.18)	95.69 (0.25)	95.10 (0.52)	94.78 (0.71)	92.75 (1.10)	19.73***	Class 1 vs. 3* Class 1 vs 4* Class 1 vs. 5* Class 2 vs 5
# Office Discipline Referrals	0.36 (0.09)	0.79 (0.16)	1.00 (0.26)	4.13 (0.95)	4.87 (1.13)	44.87***	Class 1 vs. 2* Class 1 vs. 3* Class 1 vs 4* Class 1 vs 5* Class 2 vs 4 Class 2 vs. 5* Class 3 vs 4 Class 3 vs 5
# In-School Suspension	0.09 (0.03)	0.17 (0.04)	0.17 (0.06)	0.91 (0.24)	1.23 (0.36)	27.82***	Class 1 vs. 4* Class 1 vs. 5 Class 2 vs 4 Class 2 vs 5 Class 3 vs 4 Class 3 vs 5
# Out-of-School Suspension	0.02 (0.10)	0.06 (0.02)	0.01 (0.02)	0.23 (0.07)	0.31 (0.15)	16.55**	Class 1 vs. 4** Class 2 vs. 3* Class 2 vs 4* Class 3 vs 4** Class 3 vs 5*
Reading Scores	60.78 (1.47)	46.87 (1.71)	42.66 (3.29)	33.75 (4.34)	23.10 (4.48)	115.10***	Class 1 vs. 2* Class 1 vs. 3* Class 1 vs 4* Class 1 vs 5* Class 2 vs 4 Class 2 vs 5* Class 3 vs 5**
Math Scores	60.28 (1.51)	45.59 (1.75)	35.06 (3.20)	29.16 (3.88)	19.62 (4.52)	156.64***	Class 1 vs. 2* Class 1 vs. 3* Class 1 vs 4* Class 1 vs 5* Class 2 vs. 3 Class 2 vs 4* Class 2 vs 5* Class 3 vs 5

Chi-square p-values: *p < .05. **p < .01. ***p < .001

Social, Emotional, and Behavioral Outcomes

The Co-occurring class had the highest risk of being two standard deviations above their peers for externalizing problems in sixth grade (i.e., high externalizing problems), with 42% of these youth falling into this category. Furthermore, these youth had significantly greater likelihood of having high externalizing problems in sixth grade compared to youth in the No Risk class (odds ratio [OR] = 55.56; confidence interval [CI]: 18.51–166.67), the Low Risk class (OR = 11.63; CI: 4.69–28.57), the High Externalizing class (OR = 3.89; CI: 1.30–11.63), and the Internalizing and Emotion Dysregulation class (OR = 8.33; CI: 2.83–25.00). Youth in the High Externalizing class were significantly more likely to have high externalizing problems in sixth grade than youth in the No Risk class (OR = 14.13; CI: 4.98–40.04) and the Low Risk class (OR = 2.98; CI: 1.33–6.67). Youth in the Internalizing and Emotion Dysregulation class were significantly more likely to have high externalizing problems in sixth grade compared to the No Risk class (OR = 6.58; CI: 2.28–20.00). Finally, youth in the Low Risk class were significantly more likely to report high externalizing problems in sixth grade than the No Risk class (OR = 4.74; CI: 1.84–12.21).

Youth in the Internalizing and Emotion Dysregulation class were significantly more likely to report high internalizing symptoms in sixth grade than the No Risk class (OR = 6.71; CI: 2.89–15.63). In addition, youth in the Low Risk class had a significantly greater likelihood of high internalizing scores than those in the No Risk class (OR = 3.58; CI: 1.69–7.63). No other comparisons were significant for high internalizing problems in sixth grade.

With regard to peer relationship problems, youth in the Internalizing and Emotion Dysregulation class were significantly more likely of having high peer relationship problems in sixth grade than youth in the No Risk class (OR = 9.80; CI: 1.34–27.78) and the Low Risk class (OR = 2.16; CI: 1.01–5.38). In addition, youth in the High Externalizing class were significantly more likely to be in the high peer relationship problems than youth in the No Risk class (OR = 3.88; CI: 1.08–13.89). Youth in the Co-occurring class were more likely to report high peer relationship problems than youth in the No Risk class (OR = 6.29; CI: 1.34–29.41). Finally, in comparison to youth in the No Risk class, youth in the Low Risk class were also more likely to report high peer relationship problems in sixth grade (OR = 4.50; CI: 1.56–12.99). No other class comparisons were statistically significant.

Youth in the High Externalizing class were more likely to be in the high-risk emotion dysregulation group than students in the No Risk class (OR = 7.58; CI: 2.70–17.24). Youth in the High Internalizing and Emotion Dysregulation class had significantly greater likelihood of having high emotion dysregulation problems in sixth grade than students in the No Risk class (OR = 12.20; CI: 5.18–28.57) and the Low Risk class (OR = 3.03; CI: 1.51–6.06). Youth in the Co-occurring class had a greater likelihood of reporting high emotion dysregulation problems in sixth grade than students in the No Risk class (OR = 10.31; CI: 3.21–33.33). Youth in the Low Risk class were more likely to have high emotion dysregulation problems than those in the No Risk class (OR = 4.30; CI: 1.80–9.01). No other comparisons were significant.

Youth in the High Externalizing class were more likely to have high attention and academic problems in sixth grade than youth in the No Risk class (OR = 8.13; CI: 2.44–27.12). Youth in the High Internalizing and Emotion Dysregulation class were also more likely to have high attention and academic problems in sixth grade than students in the No Risk class (OR = 8.40; CI: 2.69–26.32) and the Low Risk class (OR = 2.16; CI: 1.01–5.38). Youth in the Co-occurring class were more likely to have high attention and academic problems than youth in the No Risk class (OR = 26.36; CI: 6.67–100) and the Low Risk class (OR = 5.32; CI: 1.69–16.67). Youth in the Low Risk class were significantly more likely to have high attention and academic problems than those in the No Risk class (OR = 4.80; CI: 1.77–13.54). No other comparisons were significant.

Academic, Attendance, and Disciplinary Outcomes

In addition to student self-report, we examined whether profiles predicted the percent of days attended, number of disciplinary actions, and reading and math scores in sixth grade. The Co-occurring class had the lowest average percent days of attendance (M = 92.75), and this was significantly lower than the No Risk class (M = 96.23; χ² = 10.27, p < .001) and the Low Risk class (M = 95.69; χ² = 6.94, p < .01). In addition, the No Risk class had a higher percentage of days in attendance than the High Internalizing and Dysregulation class (M = 95.10; χ² = 4.65, p < .05) and the High Externalizing class (M = 94.78; χ² = 4.35, p < .05).

The Co-occurring class (M = 4.87) and High Externalizing class (M = 4.13) had a higher number of office discipline referrals in sixth grade. They were not statistically different from one another, but both had higher number of office discipline referrals than the three other classes. The No Risk class (M = 0.36) also had fewer office discipline referrals than the Low Risk class (M = 0.79; χ² = 4.53, p < .05) and the High Internalizing and Dysregulation class (M = 1.00; χ² = 5.38, p < .05). Similarly, the Co-occurring class (M = 1.23) and High Externalizing class (M = 0.91) had a high number of in-school suspensions in sixth grade. They were not statistically different from one another, but both had more in-school suspensions than the three other classes. With regard to out-of-school suspensions in sixth grade, again, the classes with the highest number were the Co-occurring class (M = 0.31) and the High Externalizing class (M = 0.23). The High Internalizing and Emotion Dysregulation class (M = 0.01) had the lowest average number of out-of-school suspensions in sixth grade, and this was significantly lower than the Co-occurring class (χ² = 3.92, p < .05) and the No Risk class (M = 0.02; χ² = 4.66, p < .05). The High Externalizing class had significantly more out-of-school suspensions than the No Risk class (χ² = 8.23, p < .01), the Low Risk class (M = 0.06; χ² = 4.69, p < .05), and the High Internalizing and Dysregulation class (χ² = 8.46, p < .01).

The No Risk class (M = 60.78) had the highest average reading score of all groups, and this was significantly higher than that in all other classes (p < .001). Furthermore, the Low Risk class (M = 46.87) had significantly higher reading scores than the High Externalizing class (M = 33.75; χ² = 7.23, p < .01) and Co-occurring class (M = 23.10; χ² = 24.72, p < .001). Finally, the High Internalizing and Dysregulation class (M = 42.66) had significantly higher reading scores than the Co-occurring class (χ² = 12.10, p < .01).

Similarly, the No Risk class (M = 60.28) had the highest average math score of all groups, and this was significantly higher than all other classes (p < .001). The Low Risk class (M = 45.59) had significantly higher math scores than the High Internalizing and Dysregulation class (M = 35.06; χ² = 7.89, p < .01), the High Externalizing class (M = 29.16; χ² = 13.57, p < .001), and the Co-occurring class (M = 19.62; χ² = 28.90, p < .001). Finally, the High Internalizing and Dysregulation class (M = 35.06) had significantly higher math scores than the Co-occurring class (χ² = 7.59, p < .01).

Discussion

We found evidence of five social, emotional, behavioral, and academic risk profiles in a community sample of 1,785 third graders. The vast majority of youth (83%) fell into one of two adaptive patterns characterized by average (low) or below-average (no) risk across all five risk indicators, respectively. This is in line with MTSS and public health models which predict that about 80% of youth in school will present without risk and will benefit from universal supports (see Stormont et al., 2012). Remaining youth fell into one of three elevated risk patterns marked by high externalizing (7%), high internalizing and emotional dysregulation (8%), or high co-occurring (2%) indicators. The five classes were distinguished by external criteria measured 3 years later that corresponded with their baseline risk characteristics, thus providing compelling evidence in support of the five latent risk profiles. Even the two normative groups differed on these criteria such that the no-risk group had significantly higher academic performance and lower office discipline referrals in sixth grade than the low-risk group.

The Co-occurring class had strikingly worse outcomes on nearly all sixth-grade measures compared to the other classes. In fact, the Co-occurring class was more likely to self-report high externalizing problems by a factor of 55, more likely to have attention and academic issues by a factor of 26, have high emotion dysregulation by a factor of 10, and to have peer relationship problems by a factor of 6 in sixth grade compared to their peers in the No Risk class. This is similar to prior research showing that early co-occurring problems are associated with high rates of poor academic, social, emotional, and behavioral outcomes later in development (Darney et al., 2013; Reinke et al., 2008). The High Externalizing class continued to have high externalizing problems in sixth grade but also was more likely to have high emotion dysregulation, peer relationship problems, and attention and academic problems. The High Internalizing and Emotion Dysregulation class continued to have high internalization and emotion dysregulation problems in sixth grade, and also high externalizing problems emerged in comparison to the no-risk group. The findings call attention to the enduring detrimental effects of early-onset co-occurring social, emotional, behavioral, and academic risks. Consistent evidence shows that most of the youth with these co-occurring risk characteristics will go on to have persistent failures and struggles throughout their academic careers (Darney et al., 2013; Moffitt, 1993; Reinke et al., 2008). These youth are readily identifiable with measures like the EIS.

The findings are also consistent with the developmental cascade model which predicts that long-term deleterious effects of early-onset externalizing problems evolve into multiple problem areas over time (Masten & Cicchetti, 2010). We also found evidence that youth with predominate externalizing problems continue to experience high externalizing problems at middle school entry. Furthermore, the Co-occurring and High Externalizing classes were associated with more office discipline referrals and in-school suspensions. Previous studies showed cascades for early externalizing problems lead to dual failure (e.g., poor academic performance and social competence), which predicts the development of internalizing problems and an increase in externalizing problems (X. Chen et al., 2010; Obradović et al., 2009; van Lier et al., 2012). Consistent with dual failure, youth in the High Externalizing class had significantly worse academic achievement than youth in the No and Low Risk class. However, we did not find evidence that youth in this class experienced an escalation of internalizing systems by sixth grade. Instead, only youth in the predominately internalizing class in third grade continued to be most at risk for internalizing problems 3 years later. This finding is consistent with prior evidence showing that academic and social skill deficits at school entry are significant risks for stable depressive symptoms prior to adolescence (Herman et al., 2008, 2021). Moreover, one study showed that when controlling for inattention in young children, early-onset conduct problems did not predict depressive symptoms (Herman & Ostrander, 2007). That said, externalizing problems emerged in sixth grade for the high internalizing and emotion dysregulation group. This is similar to prior research which found heterotypic and homotypic continuity of symptoms over time (Reinke & Ostrander, 2008). Interestingly, the High Internalizing and Emotion Dysregulation class had the lowest number of out-of-school suspension than any other group, including the No Risk class. This is not entirely surprising given that youth with heightened anxiety and depression internalize their behaviors and may not overtly behave in a manner that leads to exclusionary discipline for schools.

Findings point to the benefit of identifying youth at third grade with co-occurring problems, as these problems likely compound over time. Identifying profiles of risk and implementing comprehensive and tiered supports may mitigate these outcomes. These supports include implementing integrated interventions for academic and behavioral difficulties to prevent or blunt the co-occurring problem in middle school. Furthermore, youth across all risk profiles in third grade should receive supports to learn to self-regulate emotions. There is evidence that internalizing, externalizing, and comorbid psychopathologies are characterized by underlying deficits in emotion dysregulation, which likely contribute to many problematic behaviors in later life (Marmorstein & Iacono, 2003). Emotion dysregulation can lead to angry outbursts, anxiety, depression, substance abuse, suicidal thoughts, self-harm, and other self-damaging behaviors. Thus, identifying youth early who would benefit from learning coping and replacement behavior strategies can prevent a host of negative long-term trajectories.

A noteworthy feature of the present study compared to prior work is that we relied on student self-report of their social, emotional, behavioral, and academic problems in third grade as indicators in our LPAs. The findings support the idea that youth at this age (8 or 9 years old) are able to provide meaningful and accurate reports of these problem areas. The risk profiles that emerged were largely consistent with theory and prior studies. Moreover, the findings that these profiles predicted self-perceptions and school record data 3 years later in expected ways provide evidence of the veracity of these self-reports.

It is also noteworthy that there were two normative classes that also differed on outcomes in sixth grade. In the present paper, we found evidence that a higher percentage of youth in the low-risk group identified with externalizing problems, peer relationship problems, emotional dysregulation, and attention and academic problems in sixth grade. These distal outcomes may emerge due to the difficulties youth face as they transition from elementary to middle school (Reinke et al., 2008). Even youth in the typical range on internalizing and externalizing symptoms may struggle over time, thus the need for ongoing social-emotional and behavioral screening. This also speaks to the prevention paradox (i.e., that most new cases over time come from the lower-risk group because of their size; Skog, 1999). Even if only 10% of those in the low-risk groups (say of 1,000 people) go on to develop depression in the future versus 50% in a high-risk group (say of 100 people), the total number of people developing depression form the low-risk group (100) will be twice as large as the high-risk group (50).

Implications

Schools implementing universal social, emotional, and behavioral screening using a comprehensive assessment such as the EIS can identify students struggling across multiple risk areas and intervene accordingly. The EIS system provides reports back to schools that problem-solving teams can use to sort and identify students as early as third grade by self-report with profiles of co-occurring behaviors (see Reinke et al., 2018). Problem-solving teams can then provide behavior and academic supports to address the multiple areas of concern. Identifying and intervening with students in a comprehensive manner (targeting multiple risk areas at once) is important. Having screening systems provide school teams with meaningful data related to co-occurring profiles is needed. With ongoing progress monitoring and screening each school year, we may be able to mitigate later academic and behavior issues for these youth.

Limitations

The study represented a longitudinal correlational study design so causal inferences are not warranted. Experimental studies where preventive interventions are implemented to alter student risk profiles are needed to establish causality. In addition, the students were all part of a broader mental health initiative in their schools, so some may have received interventions. However, we do not have data on who in the sample received an intervention, what type of intervention, or if it was done with fidelity. This is a significant limitation. That being said, schools often implement dozens of strategies and interventions, and these are often not well documented in other school-based research studies. What is known is that the school problem-solving teams were not looking at these data with the eye toward co-occurring risk and therefore were not specifically intervening to mitigate risk using this lens. Another limitation is that the attention and academic problems subscale had lower internal consistency (slightly below .70). This would decrease the ability to detect statistically significant findings on other study variables, which was not the case. Finally, it is important to note the study occurred in specific context of schools in the middle of the country, and thus, it is not known how findings will generalize to other settings. However, a strength of the study is that the sample included schools ranging from a variety of contexts from suburban to rural settings and with a fair amount of variable in school sizes and student demographics.

Conclusion

Supporting youth with co-occurring externalizing, internalizing, emotion dysregulation, and attention and academic issues as early as third grade could prevent them from the later onset of social-emotional, behavioral, and academic difficulties during the transition to middle school when they need to navigate whole new sets of expectations and skills to be successful in middle and high school. Using universal screening to identify and comprehensively intervene with students across multiple areas of improvement (i.e., academics, self-regulation, social competence) could potentially lessen the likelihood of later ingrained and more severe problems.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The research reported here was supported by the Institute of Educational Sciences, U.S. Department of Education, through a Grant R305C190014 to the University of Missouri and by the Boone County Children Service Fund. The opinions expressed are those of the authors and do not represent views of the Institute or the U.S. Department of Education.

ORCID iD

Wendy M. Reinke

References

Bozdogan

. (1987). Model selection and Akaike’s Information Criterion (AIC): The general theory and its analytical extensions. Psychometrika, 52(3), 345–370. https://doi.org/10.1007/BF02294361

Burt

K. B.

Roisman

G. I

. (2010). Competence and psychopathology: Cascade effects in the NICHD Study of Early Child Care and Youth Development. Development and Psychopathology, 22(3), 557–567. https://doi.org/10.1017/S0954579410000271

Burt

S. A.

Clark

D. A.

Gershoff

E. T.

Klump

K. L.

Hyde

L. W.

(2021). Twin differences in harsh parenting predict youth’s antisocial behavior. Psychological Science, 32(3), 395–409. https://doi.org/10.1177/0956797620968532

Calkins

S. D.

Perry

N. B.

(2016). The development of emotion regulation: Implications for child adjustment. In Cicchetti

(Ed.), Developmental psychopathology: Maladaptation and psychopathology (3rd ed., pp. 187–242). John Wiley & Sons, Inc. https://doi.org/10.1002/9781119125556.devpsy306

Carpenter

J. L.

Drabick

D. A.

(2011). Co-occurrence of linguistic and behavioural difficulties in early childhood: A developmental psychopathology perspective. Early Child Development and Care, 181(8), 1021–1045. https://doi.org/10.1080/03004430.2010.509795

Chen

C. C.

Cheng

S.-L.

Rudasill

Senter

Zhang

Washington-Nortey

Adams

(2022). Transactions between problem behaviors and academic performance in early childhood. International Journal of Environmental Research and Public Health, 19(15), 9583. https://doi.org/10.3390/ijerph19159583

Chen

C. C.

Farmer

T. W.

Hamm

J. V.

Brooks

D. S.

Lee

Norwalk

. . .Rizzo

(2020). Emotional and behavioral risk configurations, students with disabilities, and perceptions of the middle school ecology. Journal of Emotional and Behavioral Disorders, 28(3), 180–192. https://doi.org/10.1177/1063426619866829

Chen

Huang

Chang

Wang

(2010). Aggression, social competence, and academic achievement in Chinese children: A 5-year longitudinal study. Development and Psychopathology, 22(3), 583–592. https://psycnet.apa.org/doi/10.1017/S0954579410000295

Chow

J. C.

Wehby

J. H.

(2018). Associations between language and problem behavior: A systematic review and correlational meta-analysis. Educational Psychology Review, 30, 61–82. https://psycnet.apa.org/doi/10.1007/s10648-016-9385-z

10.

Cicchetti

Toth

S. L.

(1998). The development of depression in children and adolescents. American Psychologist, 532(2), 221–241. https://doi.org/10.1037/0003-066X.53.2.221

11.

Cosgrove

V. E.

Rhee

S. H.

Gelhorn

H. L.

Boeldt

Corley

R. C.

Ehringer

M. A.

Young

S. E.

Hewitt

J. K.

(2011). Structure and etiology of co-occurring internalizing and externalizing disorders in adolescents. Journal of Abnormal Child Psychology, 39(1), 109–123. https://doi.org/10.1007/s10802-010-9444-8

12.

Costello

E. J.

Copeland

Angold

(2011). Trends in psychopathology across the adolescent years: What changes when children become adolescents, and when adolescents become adults? Journal of Child Psychology and Psychiatry, 52(10), 1015–1025. https://doi.org/10.1111/j.1469-7610.2011.02446.x

13.

Darney

Reinke

W. M.

Herman

K. C.

Stormont

Ialongo

(2013). Children with co-occurring academic and behavior problems in 1st grade: Distal outcomes in 12th grade. Journal of School Psychology, 51, 117–158. https://doi.org/10.1016/j.jsp.2012.09.005

14.

Ferguson

S. L. G.

Moore

E. W.

Hull

D. M.

(2020). Finding latent groups in observed data: A primer on latent profile analysis in Mplus for applied researchers. International Journal of Behavioral Development, 44(5), 458–468. https://doi.org/10.1177/0165025419881721

15.

Herman

K. C.

Cohen

Reinke

W. M.

Ostrander

Burrell

McFarlane

Duggan

A. K.

(2018). Using latent profile and transition analyses to understand patterns of informant ratings of child depressive symptoms. Journal of School Psychology, 69, 84–99. https://psycnet.apa.org/doi/10.1016/j.jsp.2018.05.004

16.

Herman

K. C.

Lambert

S. F.

Reinke

W. M.

Ialongo

N. S.

(2008). Low academic competence in first grade as a risk factor for depressive cognitions and symptoms in middle school. Journal of Counseling Psychology, 55(3), 400. https://doi.org/10.1037/a0012654

17.

Herman

K. C.

Ostrander

(2007). The effects of attention problems on depression: Developmental, academic, and cognitive pathways. School Psychology Quarterly, 22(4), 483. https://psycnet.apa.org/doi/10.1037/1045-3830.22.4.483

18.

Herman

K. C.

Reinke

W. M.

Huang

F. L.

Thompson

A. M.

Doyle-Baker

(2021). Investigating the psychometric properties of the Early Identification System—Student Report in a middle school sample. School Psychology, 36, 34–46. https://doi.org/10.1037/spq0000414

19.

Herman

K. C.

Reinke

W. M.

Thompson

Huang

Owens

(2023). Usability and social consequences of the Early Identification System as a universal screener for social, emotional, and behavioral risks. School Psychology, 38, 148–159. https://psycnet.apa.org/doi/10.1037/spq0000538

20.

Huang

Reinke

W. M.

Thompson

Herman

K. C.

& County Schools Mental Health Coalition, . (2019). An Investigation of the psychometric properties of the Early Identification System (EIS)-Student Report. Journal of Psychoeducational Assessment, 37, 473–485. https://doi.org/10.1037/spq0000414

21.

Little

R. J.

(1995). Modeling the drop-out mechanism in repeated-measures studies. Journal of the American Statistical Association, 90(431), 1112–1121. https://doi.org/10.1080/01621459.1995.10476615

22.

Mendell

N. R.

Rubin

D. B.

(2001). Testing the number of components in a normal mixture. Biometrika, 88(3), 767–778. https://doi.org/10.1093/biomet/88.3.767

23.

Marmorstein

N. R.

Iacono

W. G.

(2003). Major depression and conduct disorder in a twin sample: Gender, functioning, and risk for future psychopathology. Journal of the American Academy of Child and Adolescent Psychiatry, 42(2), 225–233. https://doi.org/10.1097/00004583-200302000-00017

24.

Masten

A. S.

Cicchetti

(2010). Developmental cascades. Development and Psychopathology, 22(3), 491–495. https://psycnet.apa.org/doi/10.1017/S0954579410000222

25.

Masten

A. S.

Roisman

G. I.

Long

J. D.

Burt

K. B.

Obradović

Riley

J. R.

. . .Tellegen

(2005). Developmental cascades: Linking academic achievement and externalizing and internalizing symptoms over 20 years. Developmental Psychology, 41(5), 733. https://doi.org/10.1037/0012-1649.41.5.733

26.

McIntosh

Goodman

(2016). Integrated multi-tiered systems of support: Blending RTI and PBIS. Guilford Press.

27.

McLachlan

G. J.

(1987). On bootstrapping the likelihood ratio test statistic for the number of components in a normal mixture. Journal of the Royal Statistical Society: Series C (Applied Statistics), 36(3), 318–324. https://doi.org/10.2307/2347790

28.

Metsäpelto

R.-L.

Zimmermann

Pakarinen

Poikkeus

A.-M.

Lerkkanen

M.-K.

(2020). School grades as predictors of self-esteem and changes in internalizing problems: A longitudinal study from fourth through seventh grade. Learning and Individual Differences, 68, 101807. https://doi.org/10.1016/j.lindif.2019.101807

29.

Moffitt

T. E.

(1993). Adolescence-limited and life-course-persistent antisocial behavior: A developmental taxonomy. Psychological Review, 100(4), 674. https://psycnet.apa.org/doi/10.1037/0033-295X.100.4.674

30.

Moilanen

K. L.

Shaw

D. S.

Maxwell

K. L.

(2010). Developmental cascades: Externalizing, internalizing, and academic competence from middle childhood to early adolescence. Development and Psychopathology, 223(3), 635–653. https://doi.org/10.1017/S0954579410000337

31.

Muthén

L. K.

Muthén

B. O.

(2010). Growth modeling with latent variable using Mplus: Advanced growth models, survival analysis and missing data. Mplus Short Courses.

32.

Nylund

K. L.

(2007). Latent transition analysis: Modeling extensions and an application to peer victimization [Doctoral dissertation]. University of California, Los Angeles.

33.

Nylund

K. L.

Asparouhov

Muthén

B. O.

(2007). Deciding on the number of classes in latent class analysis and growth mixture modeling: A Monte Carlo simulation study. Structural Equation Modeling: A Multidisciplinary Journal, 14(4), 535–569. https://doi.org/10.1080/10705510701575396

34.

Nylund

K. L.

Muthén

Bellmore

Nishina

Graham

Juvoven

(2005). The state of victimization during middle school: A latent transition mixture model approach [Paper presentation]. Annual Convention of the Society for Prevention Research, Washington, DC.

35.

Obradović

Burt

K. B.

Masten

A. S.

(2009). Testing a dual cascade model linking competence and symptoms over 20 years from childhood to adulthood. Journal of Clinical Child & Adolescent Psychology, 39(1), 90–102. https://doi.org/10.1080/15374410903401120

36.

Okano

Jeon

Crandall

Riley

(2020). Differential effects of internalizing behaviors on academic functioning for girls versus boys: An analysis of developmental cascades from elementary to high school. Development and Psychopathology, 32(2), 751–764. https://doi.org/10.1017/s0954579419000737

37.

Patterson

G. R.

Reid

J. B.

Dishion

T. J.

(1992). A social learning approach: Antisocial boys. (Vol. 4.) Castalia.

38.

Perry

N. B.

Calkins

S. D.

Dollar

J. M.

Keane

S. P.

Shanahan

(2018). Self-regulation as a predictor of patterns of change in externalizing behaviors from infancy to adolescence. Development and Psychopathology, 302(2), 497–510. https://doi.org/10.1017/S0954579417000992

39.

Ramaswamy

DeSarbo

W. S.

Reibstein

D. J.

Robinson

W. T.

(1993). An empirical pooling approach for estimating marketing mix elasticities with PIMS data. Marketing Science, 12(1), 103–124. http://dx.doi.org/10.1287/mksc.12.1.103

40.

Reinke

W. M.

Herman

K. C.

(2002). Creating school environments that deter antisocial behaviors in youth. Psychology in the Schools, 39(5), 549–559. https://doi.org/10.1002/pits.10048

41.

Reinke

W. M.

Herman

K. C.

Huang

McCall

Holmes

Thompson

Owens

(2022). Examining the factor structure and concurrent and predictive validity of the Early Identification System—Student Report in an elementary school sample. Journal of School Psychology, 90, 114–134. https://psycnet.apa.org/doi/10.1016/j.jsp.2021.10.001

42.

Reinke

W. M.

Herman

K. C.

Petras

Ialongo

(2008). Empirically-derived subtypes of child academic and behavior problems: Co-Occurrence and distal outcomes. Journal of Abnormal Child Psychology, 36, 759–777. https://doi.org/10.1007/s10802-007-9208-2

43.

Reinke

W. M.

Ostrander

(2008). Heterotypic versus homotypic continuity: The moderating effects of gender and age. Journal of Abnormal Child Psychology, 36, 1109–1121. https://psycnet.apa.org/doi/10.1007/s10802-008-9236-6

44.

Reinke

W. M.

Thompson

Herman

K. C.

Holmes

Owens

Cohen

Tanner-Jones

Henry

Green

Copeland

., & County Schools Mental Health Coalition (2018). The County Schools Mental Health Coalition: A model for community level impact. School Mental Health, 10, 173–180. https://doi.org/10.1007/s12310-017-9227-2.

45.

Renaissance Learning. (2020). Research foundation for STAR adaptive assessment [White paper]. https://doc.renlearn.com/kmnet/r001480701gcfbb9.pdf

46.

Riley

A. W.

(2004). Evidence that school-age children can self-report on their health. Ambulatory Pediatrics, 4(4), 371–376. https://doi.org/10.1367/a03-178r.1

47.

Rosato

N. S.

Baer

J. C.

(2012). Latent class analysis: A method for capturing heterogeneity. Social Work Research, 36(1), 61–69. https://psycnet.apa.org/doi/10.1093/swr/svs006

48.

Schafer

J. L.

Graham

J. W.

(2002). Missing data: Our view of the state of the art. Psychological Methods, 7(2), 147. https://psycnet.apa.org/doi/10.1037/1082-989X.7.2.147

49.

Schwartz

(1978). Estimating the dimension of a model. The Annals of Statistics, 6(2), 461–464. https://doi.org/10.1214/aos/1176344136

50.

Sclove

S. L.

(1987). Application of model-selection criteria to some problems in multivariate analysis. Psychometrika, 52, 333–343. https://psycnet.apa.org/doi/10.1007/BF02294360

51.

Shi

Ettekal

(2021). Co-occurring trajectories of internalizing and externalizing problems from grades 1 to 12: Longitudinal associations with teacher-child relationship quality and academic performance. Journal of Educational Psychology, 113(4), 808–829. https://doi.org/10.1037/edu0000525

52.

Skog

O. J.

(1999). The prevention paradox revisited. Addiction, 94(5), 751–757. https://doi.org/10.1046/j.1360-0443.1999.94575113.x

53.

Stormont

Reinke

Herman

K. C.

Lembke

(2012). Tier 2 interventions: Academic and behavior supports for students at risk for failure. Guilford Press.

54.

Thompson

Herman

K. C.

Reinke

W. M.

Huang

Eddy

Doyle-Baker

(2021). Examining the factor structure and concurrent and predictive validity of the Early Identification System–Student Report in a high school sample. School Mental Health, 13, 28–40. https://doi.org/10.1007/s12310-020-09396-5

55.

Thompson

A. M.

Reinke

W. M.

Holmes

Danforth

Herman

K. C

., & The County School Mental Health Coalition. (2017). The County School Mental Health Coalition: A model for a systematic approach to supporting youth. Children & Schools, 39, 209–218. https://doi.org/10.1093/cs/cdx019

56.

Vaillancourt

Brittain

H. L.

McDougall

Duku

(2013). Longitudinal links between childhood peer victimization, internalizing and externalizing problems, and academic functioning: Developmental Cascades. Journal of Abnormal Child Psychology, 41(8), 1203–1215. https://doi.org/10.1007/s10802-013-9781-5

57.

Van der Ende

Verhulst

F. C.

Tiemeier

. (2016). The bidirectional pathways between internalizing and externalizing problems and academic performance from 6 to 18 years. Development and Psychopathology, 28, 855–867. https://doi.org//10.1017/S0954579416000353

58.

van Lier

P. A.

Vitaro

Barker

E. D.

Brendgen

Tremblay

R. E.

Boivin

. (2012). Peer victimization, poor academic achievement, and the link between childhood externalizing and internalizing problems. Child Development, 83(5), 1775–1788. https://doi.org/10.1111/j.1467-8624.2012.01802.x

59.

Vuong

Q. H.

(1989). Likelihood ratio tests for model selection and non-nested hypotheses. Econometrica: Journal of the Econometric Society, 57, 307–333. https://doi.org/10.2307/1912557

60.

Woltering

Shi

(2016). On the neuroscience of self-regulation in children with disruptive behavior problems: Implications for education. Review of Educational Research, 86(4), 1085–1110. https://doi.org/10.3102/0034654316673722

61.

Yang

Shields

G. S.

Zhang

Chen

Romer

A. L.

(2022). Child executive function and future externalizing and internalizing problems: A meta-analysis of prospective longitudinal studies. Clinical Psychology Review, 97, 102194. https://doi.org/10.1016/j.cpr.2022.102194

62.

Zimmermann

Schütte

Taskinen

Köller

Graesser

A. C.

(2013). Reciprocal effects between adolescent externalizing problems and measures of achievement. Journal of Educational Psychology, 105(3), 747–761. https://doi.org/10.1037/a0032793