Abstract
The primary goal of the present systematic review was to examine the criteria and measures used for assessing students with specific comprehension deficit (SCD) who have adequate decoding skills but still perform poorly on reading comprehension assessments. From a systematic review of 32 studies, we found four predominant selection approaches for classifying students with SCD and a wide range of measurements of reading skills used to distinguish students with SCD from skilled readers. In addition, to develop a reading profile for students with SCD, we performed a meta-analysis to quantify the characteristics of SCD by comparing their reading skills to those of skilled readers. Results revealed that students with SCD demonstrated deficits in oral language (i.e., vocabulary and listening comprehension) and reading comprehension, despite adequate decoding and fluency skills. Their reading comprehension deficits (Hedges’s g = −3.28) were also more severe than their oral language deficits (Hedges’s g = −0.95). We provide recommendations and implications for future researchers and classroom teachers.
The newly released National Assessment of Educational Progress report (National Center for Education Statistics, 2019) demonstrated that both fourth and eighth graders’ average reading scores decreased in 2019 compared with 2017. In addition, in the United States, approximately 65% of fourth and eighth graders’ reading scores were below the proficient level. These findings indicate that a large proportion of students are likely encountering challenges in reading activities and highlight the need to identify and specify the characteristics of struggling readers, a fundamental step for developing effective differentiated instruction (Fricke et al., 2013). Within this population, relevant research further demonstrates that nearly 10% of struggling readers have adequate decoding skills but still perform poorly on reading comprehension assessments (Nation & Snowling, 2000). This group is defined as students with specific comprehension deficit (SCD, Cain, 2003; Ricketts et al., 2014). Categorizing common characteristics of children with SCD remains a challenge due to the complex nature of reading skills, variations in measurements of reading skills, and inconsistent selection criteria used across studies.
First, the complex nature of reading comprehension has resulted in challenges in identifying students with SCD. One reason for this difficulty comes from the fact that understanding written text involves a set of reading skills beyond decoding, such as oral language, higher-level thinking, inference making, and comprehension monitoring (Cain, 2006; Oakhill et al., 2005; Silva & Cain, 2015). When teachers attempt to identify students with SCD, there may be a lack of consistent conclusions regarding which specific reading skills should be included to best describe the characteristics of this population. For example, Spencer and Wagner (2018) found students with SCD are more likely to struggle with oral language skills. However, oral language can be measured by multiple subskills, such as vocabulary knowledge (Nation & Snowling, 1998), listening comprehension, and syntactic awareness (Ehrlich & Remond, 1997). This complex interrelation of reading skills brings challenges to identifying the specific skills for intervention. More consistent criteria would help teachers, school personnel, and researchers accurately identify students with SCD and understand their needs.
To complicate this issue further, existing research found that variations in measures of reading skills also result in discrepant findings of SCD (Keenan et al., 2014). Specifically, Keenan and colleagues (2014) found that “only about half the time (54%) does a comprehender who performs poorly on one type of test also perform poorly on another type” (p. 10), when using four reading comprehension assessments to identify struggling readers (i.e., Peabody Individual Achievement Test; Dunn & Markwardt, 1970; Woodcock–Johnson Passage Comprehension-III, Woodcock et al., 2001; Gray Oral Reading Test-III, Wiederholt & Bryant, 1992; Qualitative Reading Inventory-III, Leslie & Caldwell, 2001).
Discrepant findings across different assessment measures may be due to the fact that these comprehension assessments target different comprehension skills. Specifically, the Peabody Individual Achievement Test and Woodcock Johnson Passage Comprehension Test-III rely more heavily on decoding than on comprehension, as both use single sentences to measure reading comprehension and provide few context clues. Students are required to complete a blank test (e.g., cloze tests) for identifying words in a single sentence (Keenan et al., 2008). However, the other assessments (i.e., Gray Oral Reading Test-III and Qualitative Reading Inventory-III) provide longer texts, such as a paragraph or a passage. Students complete multiple-choice questions or answer open-ended questions to demonstrate comprehension skills. With varying methods for assessing reading skills, it is no surprise that different findings on identifying students would emerge. Therefore, there is a high probability that the inconsistencies of reading comprehension assessments may lead to the inconsistent diagnosis of SCD (Keenan et al., 2014). These inconsistencies may have substantial consequences such as including students in specific interventions, misusing resources, or allocating time and energy in ways that are not meeting the needs of students.
Moreover, the variations in selection criteria also contribute to the discrepant findings on students with SCD. For example, decoding can be measured through word reading, pseudoword reading, or a combination of both (see García & Cain, 2014). In a recent study, Rønberg and Peterson (2016) found that when orthographic coding is also used as a criterion for adequate word reading skills, only 0.4% to 2.2% of the participants can be defined as part of the SCD group, however, the numbers increased to 3% to 6%, when pseudoword reading was used as a criterion for adequate word reading skills. These findings reveal that using different selection criteria may affect the identification of students with SCD, and scholars’ interpretation of research findings can be inaccurate, although there is scant research drawing attention to these concerns.
In summary, previous studies have measured a wide range of reading skills that are associated with reading comprehension and adopted different assessments and criteria to identify SCD. However, there remains a dearth of research addressing concerns over how these decisions to identify SCD are quantified through measures. These variations, therefore, have left confusion and less definitive answers to the questions: What are the common characteristics of students with SCD in terms of their reading skills? What selection methods are best for identifying this group of students (Keenan et al., 2014)? To date, recent research has attempted to determine the common characteristics of students with SCD through latent profile analysis (e.g., Capin et al., 2021; Solari et al., 2019; Zajic et al., 2020). For example, Capin et al. (2021) classified a sample of fourth-grade poor comprehenders (n = 446) and found three latent profiles: (a) moderate deficits in both word reading and listening comprehension of similar severity (91%), (b) severe deficit in word reading paired with moderate listening comprehension deficit (5%), and (c) severe deficit in listening comprehension with moderate word reading difficulties (4%). These findings provided insights about the characteristics of the cognitive and reading skills of poor comprehenders. However, current research has drawn less attention to the inconsistencies of reading measurements, which may significantly affect the identification of SCD.
Therefore, this systematic review sought to synthesize the selection methods used to determine students with SCD in the existing literature, as well as develop a reading profile to describe the characteristics of students with SCD. Moreover, we aimed to determine whether the variations in selection methods may moderate the reading profile of students with SCD. This much needed research may provide more specific criteria for identifying students with SCD that can be used alongside various measurement methods to ensure students receive effective intervention and remediation.
Theoretical Framework and Literature Reviews of Reading Comprehension and SCD
As defined by the RAND Reading Study Group (2002), reading comprehension is an active process between the reader and text. To successfully comprehend text, readers need to simultaneously extract and construct meaning through this interaction. In the past few decades, an increasing number of studies have merged exploring the process of reading comprehension. Below, we discuss the supporting theory and review literature on reading skills related to comprehension and SCD, which informed our understanding of students with SCD.
Reading Skills Related to Comprehension
The Simple View of Reading (SVR) has frequently been used to explain the reading skills associated with SCD (e.g., Spencer & Wagner, 2018). According to SVR, reading comprehension is a product of decoding and oral language comprehension (Gough & Tunmer, 1986; Hoover & Gough, 1990). Furthermore, Hoover and Gough (1990) defined decoding as “efficient word recognition” (p. 130) and listening comprehension as “the ability to take lexical information and derive sentence and discourse interpretations” (p. 131). Both components (decoding and listening comprehension) are equally important because deficits in either may result in comprehension struggles. Over the past three decades, numerous empirical studies have shown that these two factors account for a large proportion of variance in reading comprehension (e.g., Catts et al., 2005; Georgiou et al., 2009; Kirby & Savage, 2008), with an estimation range from 40% to 80% (Kendeou et al., 2009). However, the unexplained variance still exists, which leads to the debate about further revisions on the SVR framework.
Among all the attempts to determine reading skills that contribute to comprehension, some scholars raise concerns about whether the two factors (i.e., decoding and listening comprehension) are accurately designated. Based on SVR’s original conceptualization, listening comprehension is supposed to embrace all linguistic knowledge in oral language (Gough & Tunmer, 1986). However, to better predict reading comprehension, the inclusion of listening comprehension in measures is not sufficient to explain the variance. Multiple underlying skills should be considered, such as vocabulary knowledge (Braze et al., 2007), syntactic awareness (Catts et al., 2006; Foorman et al., 2015), and morphological awareness (Carlisle, 2000; O’Reilly et al., 2012). For example, vocabulary knowledge and syntactic awareness have a unique contribution to reading comprehension after controlling for listening comprehension (Foorman et al., 2015; Ouellette & Beers, 2010). In recent research on poor comprehenders, Spencer and Wagner (2018) used the term, oral language, to replace listening comprehension in the SVR framework. This construct synthesizes all the skills mentioned above and better demonstrates the specific comprehension-related characteristics. Based on this previous body of work and Spencer and Wagner’s (2018) systematic review, we use the term “oral language” rather than “listening comprehension” in this study to develop a reading profile for students with SCD.
Moreover, one of the criticisms related to the SVR is whether additional reading skills should be included within the model and whether the roles of these skills should be further examined (Adlof et al., 2006; Kirby & Savage, 2008). Several studies have suggested that fluency should be included as an independent construct rather than a subcomponent of decoding (Kirby & Savage, 2008; Pikulski & Chard, 2005). Evidence has found that fluency forms a bridge between decoding and reading comprehension (Pikulski & Chard, 2005) and may serve as a mediator within the modeling (Li & Wu, 2015; Silverman et al., 2013). Specifically, the three components of fluency (i.e., accuracy, speed and prosody), all have been found to share a reciprocal relationship with reading comprehension (Klauda & Guthrie, 2008). Therefore, the contribution of fluency to reading comprehension is unique. Moreover, Álvarez-Cañizo and colleagues (2015) found that even when they control for decoding and listening comprehension, deficits in fluency may still result in comprehension deficits. As such, in this systematic review, we include fluency as an independent construct in addition to decoding.
In summary, informed by prior research and the SVR framework, we examined students’ word-level reading skills and oral language skills to develop a reading profile for students with SCD. Following the example of Spencer and Wagner (2018), we used the term oral language to encompass all the comprehension-related skills.
The Previous Review of Students With SCD
In a recent meta-analytic study, Spencer and Wagner (2018) examined comprehension gaps for students with SCD by comparing them with age-matched average readers. To develop a profile for students with SCD, they synthesized a total of 86 studies that (a) reported original quantitative data; (b) measured at least one of the reading skills (i.e., reading comprehension, decoding, and oral language); (c) focused on native speakers ages 4 to 12 years; (d) included students with SCD based on their comprehension and decoding abilities; and (e) included a typically developing group of readers for comparison. Results demonstrated that students with SCD had deficits in oral language (Cohen’s d = −0.78), but these deficits were less severe than their reading comprehension difficulties (Cohen’s d = −2.78).
Although this meta-analytic study provides insights regarding characteristics of students with SCD, limited information was provided regarding the selection criteria for students with SCD. Therefore, in this study, we attempted to address this research gap through a systematic synthesis of the variations and potential inconsistencies in the selection method, especially the cut-off criteria that were used to distinguish students with SCD and skilled readers. We also examined whether selection methods have a moderating effect on the presentation of reading profiles for students with SCD. In other words, we explore what selection methods were used to identify students with SCD and whether those various selection methods resulted in different reading profiles for this group. Our review builds on the work of Spencer and Wagner (2018) by including these two important components, which were not part of the original work.
Moreover, in Spencer and Wagner’s study, it should be noted that decoding and fluency were not clearly distinguished from each other. Despite the strong correlation between decoding and fluency, previous research has highlighted the unique contribution of fluency to reading comprehension, and therefore, decoding and fluency should be considered as two distinctive reading skills (Silverman et al., 2013). Specifically, decoding is typically assessed by word reading, pseudoword reading tasks (e.g., word or pseudoword identification, word attack), or both. However, the measures of fluency involve assessing automaticity, “the speed and accuracy of reading pseudowords, words, and connected text” (Silverman et al., 2013, p. 111). Considering that fluency and decoding are often measured through different approaches, it is clear that they represent different skills. As such we consider the two constructs as distinctive reading skills in this study and reported more specific findings of the two reading skills to develop a detailed profile for students with SCD.
The Present Study
Informed by previous research and syntheses, the purpose of the present study is to further examine the identification methods of students with SCD. First, we synthesized the selection methods of students with SCD and measures of reading skills in our article sample. Second, we developed a reading profile that demonstrates the characteristics of students with SCD by comparing them with age-matched skilled readers through a meta-analytic method. Our review extends prior research as we focused more specifically on the screen assessments used to identify students with SCD and provides more convergence regarding the reading characteristics that reflect the true nature of students with SCD. Specifically, this study was guided by the following research questions and hypotheses:
Method
Searching Process and Selection Criteria
We searched articles and dissertations published between 1988 and 2018. The initial search was conducted in four databases (i.e., Eric, PsycINFO, Education Source, and ProQuest) and 10 peer-reviewed journals (i.e., Reading and Writing: An Interdisciplinary Journal, Journal of Learning Disabilities, Scientific Studies of Reading, Annals of Dyslexia, Journal of Educational Psychology, British Journal of Educational Psychology, International Journal of Language & Communication Disorders, Journal of Experimental Child Psychology, Journal of Research in Reading, and Contemporary Educational Psychology). We used the key terms “reading deficit,” “poor comprehender,” “struggling comprehender,” “low comprehender,” and “less skilled comprehender,” in a Boolean combination with terms of reading skills (e.g., “decoding,” “vocabulary,” “reading comprehension,” “oral language,” “fluency”). During the initial search, 3,097 articles were located.
We applied the following inclusion criteria for each study: (a) included screening tests of reading comprehension and other reading skills; (b) included K–12 students with SCD as a target group and skilled readers as a comparison group; (c) reported the selection criteria, assessments, and cut-off values that were used to determine students with SCD and skilled readers; (d) had the two groups matched on age and decoding and/or reading accuracy, but significantly differed in reading comprehension; (e) assessed participants in their first language; and (f) reported sufficient quantitative information that allows for the computation of effect size. We removed the duplicates and excluded articles that did not meet the inclusion criteria by reading the abstract of each article. A total of 192 articles were retained for a full review.
Using the same inclusion criteria mentioned above, we read the full text of the 192 articles and excluded 163 articles during this phase. The full-text screen yielded a total of 29 articles in our final corpus. Three articles included more than one study that met our inclusion criteria. After checking the independency of samples in these studies, we included all three studies bringing our total to 32 studies in the present review.
Coding Process and Interrater Reliability
We conducted a two-step coding process. The first and second authors independently coded 30% of the samples using a pre-constructed coding scheme. Then we discussed ambiguous items and established a consistent coding scheme that included both qualitative and quantitative information (see Online Appendix A). After consensus was reached, the first author coded the remaining samples. Finally, the second author double-coded the entire sample to establish the coding reliability. The overall results of the coding system yielded a high interrater agreement (>92.8% agreement). We discussed disagreement to reach 100% consensus across coding.
Synthesis of Studies
First, we synthesized samples to investigate the variations in screening methods, criteria, and assessments that were used to assess students’ reading skills. We extracted the key relevant information regarding selection methods, reading skills, and related instruments from the reviewed studies as tentative codes. Then we compared the commonalities and differences of the codes and categorized codes to generate common schemas in relation to our first research question: How are students with SCD identified across studies?
To answer our second and third research questions, we quantified the individual effect sizes for differences between students with SCD and skilled readers and calculated an overall effect size per study. Then, we conducted a meta-analysis using Comprehensive Meta-Analysis (Version 2, Borenstein et al., 2005) and R packages, Metafor (Viechtbauer, 2017) and Robumeta (Fisher & Tipton, 2017). We calculated effect sizes using Hedges’s g (Hedges, 1982), considering the correction for small sample sizes presented in this study. The average weighted effect sizes were calculated using random-effects models, which allow for differences in the treatment effect (i.e., participants are identified as SCD rather than skilled readers) across studies. We also calculated 95% confidence intervals (CIs) for all the average weighted effect sizes. Indices such as I2, which demonstrates the proportion of variance due to heterogeneity, and tau-squared, which indicates the variance of true effect sizes, were also reported. We examined publication biases using Egger test for funnel plot asymmetry (Egger et al., 1997).
In this review, there were many instances that one single study reported multiple effect sizes for a reading skill. To resolve the issue of the dependency among effect sizes, we used robust variance estimation with the small sample size correction (Hedges et al., 2010; Tipton & Pustejovsky, 2015). The robust variance estimation allows the inclusion of dependent effect sizes without requiring the covariance matrix of these effect sizes (Tanner-Smith & Tipton, 2014). The R package Robumeta (Fisher & Tipton, 2017) was used for calculating dependent effect sizes with the robust variance estimation method. We used the R package Metafor (Viechtbauer, 2017) to calculate independent effect sizes.
Results
Syntheses of Systematic Literature Review
The primary goal of the present study was to examine the criteria and measures used for assessing students with SCD. Our results revealed that the approaches to identify students with SCD and cut-off values used to distinguish them from skilled readers varied across studies.
The identification of students with SCD
We identified found four approaches for identifying students with SCD: (a) comparing students’ comprehension age with their chronological age, reading accuracy age or both (n = 13); (b) using the lowest percentiles (n = 9); (c) comparing SCD scores with the population norm (n = 6); and (d) using statistical techniques to set the cut-off value (n = 4).
The primary approach for identifying students with SCD is by comparing comprehension age with chronological age, reading accuracy age, or both. In total, 40.6% of the 32 studies applied this method. In many cases, researchers only provided an approximate cut-off value (e.g., a lower bound) instead of a specific range (e.g., a lower and upper bound) to classify students with SCD. However, using an estimated a cut-off value as a criterion may only include a very limited sample of students as SCD. For example, a student who scores slightly above this criterion may still need to be considered as SCD, resulting in limited support for this student. In addition, different studies may use a different approximation, resulting in inconsistent identifications. For example, multiple groups of researchers identified SCD as a deficit of at least 6 months between comprehension age and chronological age as well as reading accuracy age (e.g., Cain et al., 2000). However, other researchers set the criterion as a 12-month gap, which indicates a broader discrepancy (e.g., Cain, 2006). Such discrepancies across assessments and studies may result in mis-, over-, or underidentification of students with SCD.
Another prevalent approach is the use of a specified lowest percentile, as we found in nine of studies, researchers set the lowest 25th percentile as a baseline (e.g., Carretti et al., 2016). However, several exceptions did exist—two studies used the lowest 30th percentile as cut-off values (i.e., Ehrlich & Remond, 1997; Megherbi & Ehrlich, 2005). Interestingly, although this approach was easy to implement, and thus, was used in approximately one third of the 32 studies, very few studies reported the rationale for preferring a specific percentile value.
Moreover, we also found that six studies identified students with SCD by comparing their scores with population norms. For instance, Ricketts and colleagues (2007) labeled students with SCD as those who scored at least one standard deviation below the population norm (i.e., standard score <85) on the reading comprehension subtest of the Neale Analysis of Reading Ability-II (Neale, 1997), despite adequate decoding skills. Similarly, less attention was given to explain how the lower bound of these selection criteria was set up. For instance, while six studies used this method, it was unclear why one standard deviation below the population norm should indicate an SCD, particularly when the groups were matched on decoding levels.
Finally, we found four studies that defined SCD through advanced statistical analyses. For example, Cain and Oakhill (2011) “plotted the z-scores for word reading accuracy and reading comprehension and created two ‘buffer zones’ of 0.5 of a z-score” (p. 434). Through this method, students with SCD were identified as those whose reading comprehension z-scores were at least 0.5 below the overall sample, and word reading accuracy was zero or above the overall sample. Similarly, Elwér and colleagues (2015) used z-scores as cut-off values to identify students with SCD. However, different cut-off values were used in this study, as students with SCD were identified as those whose decoding z-scores are above −0.67 but reading comprehension z-scores are below −0.67. In summary, though the studies used a similar method of z-scores, each study applied the z-scores in a different way, resulting in inconsistent identifications of students with SCD.
To conclude, we found four methods for identifying students with SCD. In each of these methods, we also found inconsistent treatment of cut-off values. The scales are rarely consistent, which may result in different conclusions on students with SCD.
Reading skills and instruments
Based on our inclusion criteria, all samples should measure reading comprehension as a primary focus (n = 32) as well as measure a minimum of other reading skills. Our findings demonstrated that a total of three readings skills were measured across studies, including decoding (n = 15), fluency (n = 23), and oral language (n = 23). Although, in a majority of studies, measurements were frequently reported at the subskill level (e.g., reading accuracy, word reading), very few studies explicitly defined the reading skills (or constructs) they aimed to measure (e.g., fluency or decoding). Moreover, within each category, a variety of instruments were used to measure each subskill. A majority of studies used standardized tests or adapted versions of those tests, however, few reported the reliability of instruments. Similarly, among the studies that employed self-designed instruments, limited information was reported regarding instrument reliability and validity.
Reading comprehension
Three instruments were most frequently used to measure reading comprehension: the Neale Analysis of Reading Ability test (NARA; Neale, 1989, 1997) the Prove di Lettura MT per la scuola elementare test (MT; Cornoldi & Colpo, 1998, 2011), and the Woodcock Reading Mastery Tests (WRMT; Woodcock, 1987). Although all tests measure students’ reading comprehension at the passage level, the measurement formats are slightly different. For instance, WRMT required students to complete a cloze test while reading the passage, whereas NARA asked students to read aloud a series of short stories and answer a set of comprehension questions afterward. In terms of the MT test, unfortunately, there is a lack of information provided regarding the text format, as such we are unable to retrieve those details from samples. As a result, completing these tests may require students to apply different reading comprehension strategies, which may measure different domains of reading comprehension.
Decoding
A majority of studies performed researcher-designed instruments to measure students’ decoding skills through word reading and/or pseudoword reading tasks. Specifically, we found that researchers used various instruments to measure word reading, including word recognition (i.e., Ehrlich & Remond, 1997), word identification (e.g., Tong et al., 2011) and word search (e.g., Carretti et al., 2013). Although researchers reported that all these tasks sought to measure students’ word reading skills, it is important to note the subtle differences among the three tasks. For instance, word identification refers to the ability of children to sound out a word, whereas word recognition usually involves the ability to connect a word’s pronunciation with its meaning (Reutzel & Cooter, 2009). Again, using different measures and tasks of word reading may illuminate different skills that are not as comparable as needed to determine SCD.
Fluency
Our analysis showed that fluency was often measured through accuracy and speed tasks using the NARA test (Neale, 1997, 1989). Interestingly, we only identified one study that measured both subskills—reading accuracy and rate (or speed) (i.e., Tong et al., 2011). Most of the studies only measured reading accuracy and thus the findings could be misleading if the researchers indicated the sample was controlled for fluency. Because fluency consisted of three key components—accuracy, automaticity, and prosody—measuring either one of these components cannot account for all aspects of fluency (Klauda & Guthrie, 2008). In addition, two studies assessed students’ semantic fluency using the word association subtest of the Clinical Evaluation of Language Fundamentals–Revised (Semel et al., 1987). Unlike reading accuracy and speed tests, semantic fluency tasks require students to produce the greatest number of words in a specific category during a specified time period, which partly involves vocabulary knowledge. This type of assessment may not accurately assess reading fluency, as it relies on comprehension or knowledge generating. In summary, these findings demonstrated several limitations regarding fluency measurements across studies.
Oral language
Among the 32 studies, oral language was measured through vocabulary tests, listening comprehension tests, or both. Specifically, we found that vocabulary knowledge was measured through a variety of assessments, which targeted different vocabulary skills. For example, three groups of researchers (i.e., Cain, 2006; Cain et al., 2005; Cain & Oakhill, 2011) specified that both sight vocabulary and receptive vocabulary were assessed through different vocabulary assessments (e.g., the Gates MacGinitie Vocabulary subtest, MacGinitie et al., 2000; the British Picture Vocabulary Scale, Dunn et al., 1992). Moreover, among the five studies that measured listening comprehension, we identified that two studies focusing on either syntactic comprehension (Bonnotte & Casalis, 2010) or a cross-modal naming task (Megherbi & Ehrlich, 2005). Finally, only two studies measured both vocabulary and listening comprehension skills (i.e., Cain, 2003; Elwér et al., 2015). Similar to other reading subskills, oral language was inconsistently measured, defined, and assessed.
Developing a Reading Profile for Students With SCD Through a Meta-Analysis
To answer our second research question, we conducted a meta-analysis to quantify the characteristics of SCD by comparing the reading profiles of students with SCD to those of skilled readers. We focused on four reading skills as follows: (a) reading comprehension, (b) decoding, (c) fluency, and (d) oral language. Below we specify the effect sizes of each comparison in terms of each reading skill and reported the analyses of publication bias.
Reading comprehension
Thirty-four effect sizes of reading comprehension were extracted from 32 studies (see Table 1). The average weighted effect size of reading comprehension was large and statistically significant (Hedges’s g = −3.28, 95% CI = [−3.89, −2.68]). Consistent with our hypothesis and prior research, deficits in reading comprehension were greater than those captured by oral language measures. The test of heterogeneity suggested large variability across studies (I2 = 91.38%). Sensitivity analysis indicated the robustness of effect sizes across different ρ values. Finally, the Egger test of funnel plot asymmetry showed that the estimates were asymmetric (z = −5.42, p < .01, see Figure 1).
Average Weighted Effect Size Estimates and Heterogeneity Statistics of the Included Studies (Random-Effect Model).
Note. k = number of effect sizes; g = average weighted effect size estimate; CI = confidence interval.
p < .001.

Funnel plots for between-group comparisons.
Decoding
We extracted 19 effect sizes for decoding from 15 studies (see Table 1). The average weighted effect size of decoding was not statistically significant from zero (Hedges’s g = −0.07, 95% CI = [−0.23, 0.08]), which indicated that decoding skills of students with SCD matched that of skilled readers. A majority of studies measured decoding through either word reading (n = 5) or pseudoword reading (n = 7) tasks. Only three studies measured both word and pseudoword reading, and one of these three studies reported a composite score of the two measures. Both types of measures had statistically nonsignificant average weighted effect sizes (Hedges’s g = −0.29, 95% CI = [−0.63, 0.06] for word reading; Hedges’s g = 0.00, 95% CI = [−0.20, 0.20] for pseudoword reading).
Nearly 9.82% of the variation across studies was due to heterogeneity for decoding. The estimate was higher for word reading measures (I2 = 66.51%, Qdf=7 = 18.22, p = .01) but almost zero for pseudoword reading (Qdf=9 = 1.99, p > .05). Sensitivity analyses suggested that varying rho (ρ) values did not change the estimation of effect sizes and the result was robust. The Egger test of funnel plot asymmetry was not statistically significant (z = −1.78, p > .05, see Figure 1).
Fluency
We extracted 27 effect sizes of fluency from 23 studies (see Table 1). The average weighted effect size of fluency was also not statistically significant from zero (Hedges’s g = 0.03, 95% CI = [−0.17, 0.22]), which indicated that students with SCD were matched with skilled readers for fluency. When reporting the skills of students with SCD, most studies only included measures of reading accuracy (n = 16). However, three studies relied solely on reading speed as a control measure for students with SCD. In addition, three studies measured both reading accuracy and speed, and the remaining two studies used a composite score of the two measures. The average weighted effect sizes for reading accuracy and speed were not statistically significant (Hedges’s g = −0.04, 95% CI = [−0.21, 0.14] for reading accuracy; Hedges’s g = 0.27, 95% CI = [−0.23, 0.77] for reading speed). Variability due to heterogeneity was approximately 45.91% for fluency. The test for heterogeneity was rejected for reading accuracy (Qdf=18 = 22.68, p > .05), but not for reading speed (I2 = 67.41%, Qdf=4 = 11.96, p = .02). Sensitivity analyses indicated the robustness of effect sizes regardless of the change ρ values. The Egger test of funnel plot asymmetry did not yield publication bias (z = −1.43, p > .05, see Figure 1).
Oral language
Twenty-eight effect sizes of oral language were extracted from 23 studies (see Table 1). The average weighted effect size of oral language was moderate and statistically significant (Hedges’s g = −0.95, 95% CI = [−1.39, −0.51]). Specifically, we found that 18 studies measured vocabulary, three studies assessed listening comprehension, and two studies measured both skills. Our analysis showed that the average weighted effect size for vocabulary (Hedges’s g = −0.57, 95% CI = [−0.82, −0.32]) was smaller than that for listening comprehension (Hedges’s g = −2.72, 95% CI = [−4.05, −1.38]), although both were statistically significant.
Nearly 84.08% of the between-study variation for oral language was related to heterogeneity. The estimates were also large when testing heterogeneity for vocabulary (I2 = 66.68%, Qdf=22 = 69.50, p < .01) and listening comprehension (I2 = 92.59%, Qdf=4 = 28.63, p < .01). Sensitivity analysis did not indicate that changing ρ values resulted in variation of the observed effect sizes. The Egger test of funnel plot asymmetry for oral language was statistically significant (z = −4.10, p < .01), which suggested the existence of asymmetry in estimates (see Figure 1).
Moderator analysis
To answer the third research question, we included four selection methods for categorizing students with SCD as a moderator and examined whether different classification approaches would affect the profiles of students with SCD. Specifically, we analyzed whether using different selection methods would result in differences between students with SCD and skilled readers, based on reading skills. Interestingly, findings showed that the selection methods did not yield a statistically significant impact on the effect sizes of the four reading skills (see Table 2). In other words, the differences of students with SCD and skilled readers’ reading profiles were robust regardless of how students with SCD were grouped in a certain study.
Moderator Analyses for the Comparison Between Students With Specific Comprehension Deficit and Skilled Readers.
Note. Method 1: Compare specific comprehension deficit (SCD) students’ comprehension age and chronological age/grade level; Method 2: Use lowest percentile; Method 3: Compare students’ score with the population norm; Method 4: Apply statistical techniques. CI = confidence interval.
Discussion and Implications
Based on our synthesis and meta-analysis, we found four approaches for identifying students with SCD. We also found that each study used different subskills and instruments to determine students with SCD, making comparisons across studies more challenging. Finally, we found that SCD had comparable performance levels for decoding and fluency as their skilled reading counterparts, but their deficit in reading comprehension was more severe than that in oral language. Through these findings, we can specify a starting point for a reading profile for students with SCD. In the following sections, we discussed these findings further.
Identification of Students With SCD
Our first research question asked how students with SCD were identified across studies. To answer this question, we synthesized the selection methods used to identify students with SCD. Despite the fact that the terms, “less-skilled readers” (e.g., Cain et al., 2005), “less skilled comprehenders” (e.g., Bonnotte & Casalis, 2010), and “less-skilled readers” (e.g., Ehrlich & Remond, 1997) were interchangeably used across studies, we found researchers used different criteria to classify students with SCD as following: (a) comparing students’ comprehension age with their chronological age, reading accuracy age, or both; (b) using lowest percentiles; (c) comparing SCD scores with the population norm; and (d) using statistical techniques to set up the cut-off point. Although we acknowledge that the four methods could be useful to teachers, researchers, and policymakers for different purposes when discussing the best instructional plan for individual children, we suspect the varying approaches may yield confusion and inconsistency. This inconsistency could provide a limitation for school personnel who seek to decide how best to help specific groups of children (Lee & Tsai, 2017).
Consequently, based on the findings of our first research question, we further proposed a new hypothesis—we anticipated that selection criteria for students with SCD may moderate our findings about their reading comprehension performance. However, our meta-analytic results did not show statistically significant differences for SCD reading performances as related to the variation of approaches to identify the students with SCD. This result may be due to the small sample size. For example, we only identified four studies that use statistical techniques to set up the cut-off point. However, with a p value approaching .05 on reading comprehension, it is likely that a larger sample size may result in a stronger conclusion and provide a statistically significant result. Future research with more samples may help in addressing this issue.
Measurements of Reading Skills
In addition, we synthesized the measurements of reading skills. All the included studies in this review measured reading comprehension as the primary focus, however, we found the measurements of reading comprehension varied. Keenan and colleagues (2014) found that variations in comprehension assessments resulted in discrepant findings of SCD. For example, using a long passage and open-ended questions to assess comprehension may yield different findings on SCD, compared with the use of a cloze test, because the two tests focus on different comprehension skills (Keenan et al., 2008). Notably, we not only found variations in comprehension measurements across studies but also a wide range of measurements used for assessing other reading skills, such as fluency, decoding, and oral language. Although the small sample size impedes our examination of how these variations in measurements may moderate the identification of students with SCD, we recommend future endeavors continue the exploration of reading comprehension measurements.
Moreover, when examining the screening instruments, we also noted a limitation in reporting of reliability estimates. Although a majority of studies that used standardized measures reported the measure reliability, few reported the reliability at the level of the study. Similarly, for studies that used researcher-developed measures, very few reported reliability or pilot study information. According to the American Educational Research Association (AERA), all studies should report evidence of reliability (American Educational Research Association [AERA], 2009). This information ensures that high-quality instruments with strong evidence of reliability are used and that the results can be trusted and replicated. Our findings indicate an inherent limitation on measurement across studies on SCD.
A Reading Profile for Students With SCD
To examine whether students with SCD perform more poorly on assessments of oral language or reading comprehension, we attempted to develop a reading profile for students with SCD through a meta-analysis. Consistent with Spencer and Wagner’s (2018) findings, we found that when compared with skilled readers, students with SCD had a deficit in oral language (Hedges’s g = −0.95) but a more severe deficit in reading comprehension (Hedges’s g = −3.28), despite their adequate decoding (Hedges’s g = −0.07). This finding supports the theoretical perspective, SVR, which emphasized the important roles of decoding and oral language in reading comprehension (Gough & Tunmer, 1986; Hoover & Gough, 1990). When students have adequate decoding skills, oral language deficits could lead to reading comprehension difficulties (Allington, 2013).
Moreover, to extend Spencer and Wagner’s (2018) meta-analysis, we examined decoding and fluency as distinctive skills in our analysis. Unsurprisingly, we did not find a significant gap between SCD and skilled readers in terms of their fluency skills (Hedges’s g = 0.03). However, we found that nearly all studies focused on accuracy and/or automaticity tasks to measure students’ fluency skills, although it is well established that fluency consisted of three components (i.e., accuracy, automaticity, and prosody, National Reading Panel, 2000). Therefore, we questioned whether students with SCD would struggle with prosody, the ability to read with inflection and tone, an overlooked skill in formal fluency assessments. In fact, previous research demonstrated that children who overlay emotion to the text would exemplify a stronger understanding of the text’s meaning (Deeney, 2010). As such, we recommend that teachers and researchers assess prosody to determine if students are making meaning of the text as they read, rather than simply decoding the words with accuracy and speed (Deeney, 2010).
In terms of oral language, literacy research has long shown connections between oral language development and reading comprehension, through links of vocabulary and listening comprehension (Hart & Risley, 2003). Interestingly, our results show that students with SCD have a more severe deficit in listening comprehension (Hedges’s g = −2.72), compared with vocabulary (Hedges’s g = −0.57). Although informative, we also found very few studies explicitly defined the components of listening comprehension. In a recent study, Kim and Pilcher (2016) found that in addition to vocabulary, syntactic knowledge, comprehension monitoring, and theory of mind all directly related to listening comprehension. When connecting our findings with Kim and Pilcher’s (2016) study, we recommend future research test more specific language and cognitive skills that account for listening comprehension in the screening assessment. This may allow the field to better understand the specific strengths and needs of students with SCD. Consequently, classroom practitioners should provide instructional strategies that can improve vocabulary, syntactic knowledge, and comprehension monitoring for students with SCD. For example, Duke and colleagues (2021) framed comprehension instruction as a layered approach that includes strategies specific to reading comprehension, language development, and knowledge of text and culture. Our findings indicate that for SCD, these skills along with more practice with oral language and fluency may be relevant. In addition, our review finds that classroom teachers should be aware of implicit biases that may exist in the measures they are using and how different selection methods may provide differing results. To this effect, we recommend teachers use multiple methods and synthesize the findings or provide other data points (e.g., classroom notes), to aid in decisions about SCD.
Limitations
Our study has several limitations. Through a systematic review, we found our samples mainly focused on four major reading skills (i.e., reading comprehension, decoding, reading fluency, and oral language) and seven reading subskills (i.e., word reading, pseudoword reading, reading accuracy, speed, vocabulary, listening comprehension, and reading comprehension). Due to the small sample size (n = 32), we were not able to retrieve information regarding other language skills to develop a more comprehensive profile for SCD students, such as their syntactic awareness and morphological awareness. Moreover, we were also interested in evaluating SCD students’ cognitive skills. However, we identified that few studies reported students’ intelligence quotient (IQ; n = 6), short-term memory (n = 2), and working memory (n = 1), thus we were unable to generate a conclusion regarding their cognitive skills, which may potentially affect their reading performance (Cain, 2006). Finally, the study scope narrows our conclusion about students with SCD to those who were tested in their native language. Future investigations are needed to examine second-language learners.
Conclusion
Through a systematic review, we found a wide range of selection methods and measurements used for identifying students with SCD. However, very few studies reported the instrument reliability. We advocate for a consistent definition and rigorous process for identifying SCD. Second, our results show that students with SCD demonstrated deficits in oral language (i.e., vocabulary and listening comprehension) and reading comprehension, despite their adequate decoding and fluency skills. Their reading comprehension deficits (Hedges’s g = −3.28) were also more severe than their oral language deficits (Hedges’s g = −0.95). Finally, our meta-analytic results did not show statistically significant differences for SCD reading performances as related to the variation of approaches to identify the students with SCD. This result may be due to the small sample size. Future research is needed to reexamine this issue with larger samples.
Supplemental Material
sj-docx-1-ldq-10.1177_07319487221085277 – Supplemental material for Differentiating Reading Profiles of Children With Specific Comprehension Deficits From Skilled Readers: A Systematic Review
Supplemental material, sj-docx-1-ldq-10.1177_07319487221085277 for Differentiating Reading Profiles of Children With Specific Comprehension Deficits From Skilled Readers: A Systematic Review by Daibao Guo, Luxi Feng and Tracey S. Hodges in Learning Disability Quarterly
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
Supplemental Material
Supplemental material is available on the Learning Disability Quarterly webpage with the online version of the article.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
