How Should Reading Disabilities be Operationalized? A Survey of Practicing School Psychologists

Abstract

School psychologists’ perceptions of how reading disabilities (RD) should be operationalized were examined and compared to those of journal editorial board members in the learning disabilities field (Speece & Shekitka, 2002). Participants were practicing school psychologists drawn from the membership directory of the National Association of School Psychologists (NASP). The sample consisted of 549 participants and was generally representative of the demographic characteristics of NASP membership at large. Results indicated that over 75% of participants endorsed using treatment validity/response to intervention (RTI), cognitive processing, and phonemic awareness as components of RD operationalization. A large percentage (61.9%) also endorsed use of an IQ–achievement discrepancy criterion. Statistically significant differences were found between the endorsements of this study's participants and those in the Speece and Shekitka (2002) study, with our participants reporting higher endorsement of RTI, cognitive processing, and IQ–achievement discrepancy criteria.

Defining reading disabilities (RD) is a historically imperfect practice that is tied to the larger history of federal definitions of learning disabilities (LD). There has long been a disconnect between the conceptual and operational definitions of LD, as well as a long history of controversy regarding both (Kavale & Forness, 2000; Kaufman, Naglieri, & Hale, 2005; Reschly & Hosp, 2004). The conceptual definition of LD has remained virtually unchanged since its formalization by the U.S. Office of Education in 1977 (Kavale & Forness, 2000). However, the operational definition (i.e., classification criteria) for identifying children with LD has changed with the newly authorized Individuals with Disabilities Education Improvement Act (IDEIA, 2004). The reauthorized act explicitly states the use of an IQ–achievement discrepancy model is not required, and there is an implicit recommendation for the use of a response to intervention (RTI) approach to LD identification. Interestingly, the wording of the new law is such that no specific procedure is required. Therefore, local educational agencies have been given freedom to operationally define LD as they see fit. Perhaps now more than ever, the perceptions on how LD should be operationalized are important because of the flexibility permitted by IDEIA 2004. Perceptions of how RD, in particular, should be operationally defined are important because the vast majority of students with LD (approximately 80%) have problems in reading (Lerner, 1989). Further, the majority of extant research on LD identification has been done specifically on RD (Speece & Shekitka, 2002).

IDEIA's new wording regarding the use of a discrepancy formula for LD determination was not unexpected: Many advocacy groups and researchers concerned with RD lobbied for just such a change. This was largely due to a culmination of research that has emphasized the use of intelligence tests, and the discrepancy model in particular, as increasingly questionable in the identification of RD (Stuebing et al., 2002). Critics of the use of IQ tests and IQ–achievement discrepancies frequently identify three main problems: (1) the discrepancy model is akin to a “wait to fail” one because children only receive special services and recognition after a “severe” discrepancy between IQ and reading achievement is able to be documented (Fletcher et al., 1998); (2) a general failure to show meaningful differences between poor readers with and without IQ–achievement discrepancies (Steubing et al., 2002; Ysseldyke, Algozzine, Shinn, & McGue, 1982; for an alternative viewpoint, see Kavale, D. Fuchs, & Scruggs, 1994; Rutter & Yule, 1975); (3) and a paucity of practical, intervention–oriented knowledge gained from intelligence test results (Good, Creek, Vollmer, & Chowdrhi, 1993; Gresham & Witt, 1997; Reschly, 1997).

Though attempted for the better part of three decades, there has been little progress in narrowing the gap between the conceptual and operational definitions. Despite some consensus among professionals on the need to align the definitions (Kavale & Forness, 2000; Reschly & Hosp, 2004), heated debate remains on how LD should be measured in practice. This is particularly the case with RD. From a practical standpoint, one could argue the predominant importance of an operational definition because it is the classification criteria within such a definition that determines which children receive relevant special education services in public schools. Many such definitions have been proposed (for a list and critique see Scruggs & Mastropieri, 2002) as alternatives to an IQ–achievement discrepancy model of RD identification. Some have proposed a method using a discrepancy between reading comprehension and listening comprehension (Aaron, 1995; Stanovich, 1991). Others have suggested simply looking at reading achievement levels, either in comparison to age or grade levels (Lyon & Fletcher, 2001; Siegel, 1989) or by using a standard score cutoff (e.g., standard score = 90, Fletcher et al., 1994; standard score = 85, Dombrowski, Kamphaus, & Reynolds, 2004). Research showing students with RD to have a phonological core deficit, which differentiates them from able readers, suggests the use of approaches that test phonological awareness skills (Adams, 1990; National Reading Panel, 2000; Stanovich, 1988; Torgesen, 2002; Torgesen & Wagner, 1998). There has also been growing advocacy for the use of an RTI model in identifying RD (Case, Speece, & Molloy, 2003; Fletcher, Coulter, Reschly, & Vaughn, 2004; Fletcher & Denton, 2003; Francis, Morris, & Lyon, 2005; Fuchs & Fuchs, 1998; Gresham, 2002; Reschly, 2003).

None of the above methods make use of IQ in the determination process, but there are those that do, particularly by looking at intra–individual strengths and weaknesses in cognitive processes and interpreting them as suggestive of LD. Naglieri has promoted the Cognitive Assessment System (Naglieri & Das, 1997) as useful in determining such process deficiencies (Naglieri, 2005). Kavale and Forness (2000) stated that the fourth level of their “operational interpretation” of LD, “attempts to capitalize on the historical association between LD and psychological process deficits” (p. 251). Others promote investigation of Cattell–Horn–Carroll Narrow Abilities in relation to achievement domains by use of a cross–battery approach to using standardized intelligence and achievement tests (e.g., Flanagan, Ortiz, Alfonso, & Mascolo, 2002).

Using such examples, Speece and Shekitka (2002) asked journal editorial board members in the LD field for their opinions on the criteria that should be included in the operationalization of RD. The authors noted increasing disfavor with an IQ–achievement discrepancy approach to RD identification, but also no clear consensus among researchers or practitioners regarding alternative methods of identifying RD. Results indicated that over two thirds of journal editorial board members favored three particular criteria: cutoff scores on measures of reading achievement and phonemic awareness, as well as a treatment validity/RTI approach. Additionally, less than one third endorsed the IQ–achievement discrepancy approach. The authors also assessed opinions about the importance of the different exclusionary criteria that have existed in the federal definition of LD. The vast majority endorsed the need for exclusionary criteria in determining RD. Mental retardation, inadequate instruction, and sensory deficits were the most commonly endorsed rule outs. There were at least two timely and logical reasons for their study. The first was to help clarify a sensible and practical approach to RD identification in advance of IDEA 2004 reauthorization. Secondly, Speece and Shekitka noted that convergence between expert opinion and research provides validation for both sources.

We agree and also argue for the importance of soliciting the opinions of those charged with carrying out the RD determination process in the actual school setting. Unless the attitudes, perceptions, and beliefs of workers in the school setting are addressed, it is unlikely that even the best of advances in identification procedures will be embraced or implemented with fidelity (Mellard, Deshler, & Barth, 2004). The timing of our study is quite different than Speece and Shekitka's (2002); their study was conducted before IDEIA 2004, making it an important indicator of perceptions before possible changes. However, IDEIA's changes have left the field in flux. Although a discrepancy formula is no longer mandated (though not disallowed), no specific replacement criterion is required. Therefore, the current study surveyed school psychologists’ perceptions of classification criteria at a time when RD diagnostic procedures are less prescribed than any time other in the last three decades.

Assessing school psychologists, specifically, is considered important because of their prominent role in RD evaluations. Approximately 75% of school psychologists’ time is dedicated to the eligibility determination process for special education (Reschly & Wilson, 1995), with the majority of individuals determined eligible under the LD category. In addition, students with RD account for the largest portion of special education services; according to Lerner (1989), RD designations account for 80% of the LD population. Bramlett, Murphy, Johnson, Wallingsford, and Hall (2002) revealed that 57% of referrals to school psychologists are primarily due to reading problems. Even if reading problems are not the primary concern of some school psychologists’ referrals, they usually are at least part of referral concerns. Fish and Margolis (1988) and Nelson and Machek (in press) found that 77% and 79%, respectively, of the students referred to school psychologists had reading problems that were at least one grade level below expectation. Thus, currently and historically, school psychologists are an integral part of the RD identification process. Assessing the opinions and attitudes of these individuals will be important for the LD field as a whole, particularly within newly proposed identification systems such as RTI. Such systems will inevitably necessitate increased collaboration across the different professionals broadly subsumed within the LD field.

The purpose of the current study was twofold. First, it examined practicing school psychologists’ perceptions of the various operational components that should be used in making RD determinations in the public schools. In addition to asking what assessment criteria should be used, we also asked participants about the exclusionary criteria they believed were most important to an RD diagnosis. Consistent with the authors of the original study (Speece & Shekitka, 2002), we felt this was an important part of the operational process given that most definitions of learning, and reading, disabilities incorporate some type of exclusionary language. This allowed us to, secondly, compare our results with those of Speece and Shekitka's (2002) survey of experts as these authors have noted the need to hear from other stakeholders, such as school psychologists, on this issue.

Method

Participants

A computer–generated, randomly selected list of 1480 members of the National Association of School Psychologists (NASP) who reported working in public schools was purchased. A survey, along with a stamped, addressed return envelope was sent to these individuals during the spring of 2005. The initial response rate was 33% (N= 488). A small proportion of the returned surveys (n= 5; 0.9%) were unusable. Surveys were considered unusable if the respondent filled out less than 75% of the survey, was not a practicing school psychologist, or was retired. A follow–up postcard request for participation in the study was sent to those who had not responded to the initial request within four weeks of mailing. An additional 66 (total N= 549) participants responded, making the final response rate 37%. This response rate is consistent with other published national surveys of school psychologists, particularly with those that have incorporated only one survey mailing (e.g., Chafouleas, Riley–Tillman, & Eckert, 2003; Chafouleas, Clonan, & Vanauken, 2002; Costenbader, Swartz, & Petrix, 1992; Demaray, Schaefer, & Delong, 2003; Hutton, Dubes, & Muir, 1992; Nickerson & Zhe, 2004; Wilczynksi, Mandal, & Fusilier, 2000).

The demographic characteristics of the current sample were compared to those of 2004–2005 NASP members (Curtis, Lopez, Batsche, & Smith, 2006) to determine whether it was representative of NASP membership. No significant differences were found for gender (χ²=.38, df= 1, p >.05), age (χ²= 5.80, df= 2, p >.05), and degree (χ²=.01, df= 1, p >.05; see Table 1 for demographic data for both samples). The two samples differed with regard to ethnicity (χ²= 12.88, df= 5, p <.05). As Table 1 indicates, the sample was well–distributed with regard to geographic location.

Table 1

Demographic Information of Sample

Demographic Characteristic	Current Sample% (n)	NASP 2004–2005% (n)
Gender
Female	75.3 (411)	73.0 (1292)
Male	24.7 (135)	26.0 (455)
Age
40 years or younger	26.3 (143)	33.1 (574)
41–49	21.7 (108)	19.4 (336)
50 years or older	52.0 (269)	47.5 (824)
Ethnicity
African American	0.7 (4)	1.9 (33)
Asian American	0.5 (3)	0.9 (16)
Latino/Hispanic	1.3 (7)	3.0 (51)
Native American	0.4 (2)	0.8 (14)
White/Non–Hispanic	95.6 (522)	92.6 (1578)
Other	1.5 (8)	0.8 (13)
Degree
Nondoctoral	73.4 (302)	67.6 (1179)
Doctoral	26.6 (146)	32.4 (565)
Geographic Locale
Central	30.2
Southeastern	20.0
Northeastern	27.1
Western	22.2

Instrument

The instrument was made up of 25 items. These 25 questions were part of a larger 49 item survey which assessed additional information regarding the identification of RD among school psychologists. The additional areas surveyed included: perceptions regarding the role of intelligence testing in RD, role identification within an RTI framework, and “general RD assessment” questions (e.g., implications of IDEIA wording changes for job role, questions regarding the relevance of the RD construct, and ultimate purpose of RD assessment [i.e. classification and/or intervention]).

The items for the current effort included a mixture of 4–point Likert–scale items and completion items. Twelve of the items were designed to obtain demographic information and the remaining items followed Speece and Shekitka's (2002) original survey. This included a self assessment of knowledge of issues relating to RD classification and definitions (extremely knowledgeable, knowledgeable, somewhat knowledgeable, limited or cursory knowledge, not at all knowledgeable 1 ). Definitional items assessed the extent to which respondents agreed with different operational components of RD identification using a 5–point Likert scale (strongly disagree, disagree, agree, strongly agree, don't know). Respondents could rate as many items as high or low as they wished. The questionnaire then asked participants to list the first and second criteria they believed to be most important. Finally, respondents were asked to declare whether exclusionary criteria should be used in a definition. If they responded “yes,” they were asked to pick which criteria from the following list (they could choose as many as they liked): mental retardation, emotional/behavioral disability, sensory deficits, economic disadvantage, cultural difference, inadequate instruction, or other.

In order to finalize the survey, it was piloted using a sample of practicing school psychologists, graduate students in school psychology, and a university faculty member to identify confusing items or items subject to a variety of interpretations. Minimal wording changes were made to the original version (Speece & Shekitka, 2002) based on this piloting feedback. Specifically, in addition to the expansion of Likert choices on self–reported knowledge, the word “criterion” was changed to “cut–off”, as in: “Cutoff score on test of intelligence”, instead of, “Criterion score on test of intelligence”. Our definition of “treatment validity” differed slightly and included the term “Response to Intervention” because of the growing use of that terminology during our piloting effort:

Treatment validity (i.e., children who do not respond to well–planned and well–implemented general education reading instruction but do respond to individualized instruction should be identified as reading disabled). (Speece & Shekitka, 2002) became,

Treatment validity (Also known as Response to Intervention: Children who do not respond to generally effective regular education instruction, which is consistently monitored, and increases in its intensity in accordance with lack of student response should subsequently be identified as reading disabled).

Finally, based on piloting comments, we included a question regarding the use of curriculum based measurement (CBM), separate from its use in establishing treatment validity. All survey questions are presented in Appendix A.

Results

Operational Definitions

Table 2 summarizes the frequency of responses regarding definitional criteria. For comparison purposes, Speece and Shekitka's (2002) results are also presented. To establish whether results from the current sample differed significantly from the original study, original data were obtained from the first author of the comparison study (Speece & Shekitka, 2002), and chi–square analyses were performed on the proportion of each sample responding “strongly agree” or “agree” to each definitional criterion. Effect sizes were determined using the phi (ϕ) coefficient. Magnitude of effect sizes were interpreted using Cohen's (1988) guidelines of. 10 as a small effect,. 30 as a medium effect, and. 50 as a large effect.

Table 2

Endorsements of Specific Definitional Criteria for RD

Criterion	Strongly Agree/Agree	Speece & Shekitka	Strongly Disagree/Disagree	Speece & Shekitka	Don't Know	Speece & Shekitka
Cut off IQ score	34.3 (188)	42.0 (47)	61.2 (336)	58.0 (65)	2.0 (11)	0
Treatment validity/RTI	81.1 (445)*	67.3 (74)	16.1 (88)	32.7 (36)	2.6 (14)	1.8 (2)
Discrepancy between listening and reading comprehension	53.2 (292)	47.2 (51)	39.0 (214)	52.8 (57)	6.7 (37)	4.4 (5)
Cognitive processing difficulties	77.6 (426)*	49.5 (52)	20.2 (111)	50.5 (53)	1.6 (9)	4.6 (5)
Cut off phonemic awareness score	75.6 (415)	79.4 (85)	20.6 (113)	20.6 (22)	3.1 (17)	2.7 (3)
Cut off reading achievement score	66.5 (365)*	83.7 (87)	30.5 (167)	16.4 (17)	1.6 (9)	1.9 (2)
IQ–Achievement discrepancy	61.9 (340)*	30.2 (32)	35.0 (192)	69.8 (74)	2.4 (13)	3.6 (4)
CBM	73.3 (402)		21.2 (116)		4.4 (24)

Notes. First number in each cell represents percentage. Numbers in parentheses represent number of individuals. *Statistically significant differences between samples is represented with an asterisk (p <. 05).

Three of the definitional criteria had support from more than 75% of the school psychologists sampled: treatment validity/RTI (88.1%), cognitive processing (77.6%), and cut–off scores on measures of phonemic awareness (75.6%). Although the latter was similar to what journal editorial board members endorsed, a significantly higher proportion of school psychologists endorsed treatment validity/RTI (χ²[1, N = 599] = 11.50, p = .001, ϕ = .14) and cognitive processing (χ²[1, N = 656] = 43.78, p = .00, ϕ = .26) as useful criteria for RD diagnosis. The school psychologists’ responses to the IQ–achievement discrepancy criterion was the other area in which the current results were significantly higher than results from Speece and Shekitka's (2002) expert sample (61.9% versus 31.2%; χ²[1, N = 654] = 41.15, p = .001, ϕ = .25).

The chi–square analyses revealed one instance of a criterion from the current sample being endorsed by a significantly smaller proportion of school psychologists: cut–off scores on tests of reading achievement (66.5% versus 83.7%; χ²[1, N = 646] = 9.19, p = .002, ϕ = −.12). In addition to the seven criteria for which Speece and Shekitka (2002) sought opinions, we also asked school psychologists about the use of CBM (not as part of an RTI model) in diagnosing RD. Seventy three percent of our sample endorsed such measures.

When asked which of the definitions was the most important criterion for diagnosing students with RD, results generally followed those above: Treatment validity/RTI stood out as a clear favorite, being endorsed by about one out of every three school psychologists (32.8%). Cutoff on phonemic awareness measures (16.9%), cognitive processing (13.8%), and IQ–achievement discrepancy (10.6%) were the only other definitions to gain 10% or more of “most important” endorsements. Results for all criteria are shown in Table 3.

Table 3

Most Important Criterion for Diagnosing Students with Reading Disabilities

Criterion	Current Sample% (N)	Speece & Shekitka% (N)
Cutoff IQ Score	2.0 (11)	0 (0)
Treatment validity/RTI	32.8 (180)	15.4 (12)
Cutoff phonemic awareness score	16.9 (93)	18.0 (14)
Cognitive processing	13.8 (76)	3.9 (3)
IQ–achievement discrepancy	10.6 (58)	7.7 (6)
Cutoff on reading achievement score	4.4 (24)	24.4 (19)
Discrepancy between listening and read. comp.	3.1 (17)	6.4 (5)

Level of Expertise

In order to look at differences by respondents’ self–rated level of knowledge, a Kruskal–Wallis k–sample nonparametric procedure was run to compare self–rated “knowledgeable” and “extremely knowledgeable” (collapsed into “more knowledgeable”) responses to those from respondents who identified themselves to be “somewhat knowledgeable” or having “limited or cursory knowledge” (collapsed into “less knowledgeable”). This comparison revealed only one criterion that differed between the two groups: Treatment validity/RTI was endorsed at a significantly higher rate amongst those who rated themselves “more knowledgeable” (strongly agree/agree = 83.2%) than those rating themselves as “less knowledgeable” (strongly agree/agree = 71.1%) in the identification RD (χ²[1, N=541] = 6.77, p=. 009, ϕ=. 11). Table 4 shows results broken down by level of expertise.

Table 4

Comparison of Endorsements Based on Self–Rated Level of Knowledge

Criterion	More Knowledgeable (%)	Less Knowledgeable (%)
Cutoff IQ Score	34.6	32.5
Treatment validity/RTI	83.2	71.1
Cutoff phonemic awareness score	75.6	77.1
Cognitive processing	77.8	74.7
IQ–achievement discrepancy	60.4	69.9
Cutoff on reading achievement score	67.0	65.1
Discrepancy between listening and read. comp.	53.7	50.6

Exclusionary Criteria

Eighty–four percent of the respondents agreed that some type of exclusionary criteria should be part of the RD classification process. The most commonly endorsed criteria were inadequate instruction (67.6%) and mental retardation (66.5%). These were also the top two exclusionary criteria endorsed in Speece and Shekitka's (2002) study. The remainder of the exclusionary criteria was endorsed as follows: cultural differences (24.4%), sensory deficits (29.3%), emotional and behavioral disability (26.2%), economic disadvantage (25.5%), and “other” (13.3%). The most commonly cited specific “other” exclusionary criterion was “poor attendance.” See Table 5 for a comparison of results on the endorsement of exclusionary criteria between the current study and Speece and Shekitka's (2002) study.

Table 5

Proportion and Number of Respondents Endorsing Specific Exclusionary Criteria

Criterion	Current Sample% (N)	Speece & Shekitka% (N)
Mental Retardation	66.5% (365)	72.2% (57)
Emotional/Behavioral Disability	26.2% (144)	29.1% (23)
Sensory Deficits	29.3% (161)	60.8% (48)
Economic Disadvantage	25.5% (140)	19.0% (15)
Cultural Differences	34.4% (189)	24.2% (19)
Inadequate Instruction	67.6% (371)	60.8% (48)
Other	13.3.% (73)	13.9% (11)

Discussion

The purpose of the current study was to assess the professional opinions of practicing school psychologists as to the operational definitions, including exclusion criteria, that should be used in making RD determinations in the public schools, and to compare such results with those of Speece and Shekitka's (2002) survey of experts.

Three of the definitional components received very strong support: treatment validity/RTI, cognitive processing, and phonemic awareness. Treatment validity/RTI received the highest endorsement, and the percentage of our participants endorsing this definitional component was significantly higher than the percentage of participants endorsing it in Speece and Shekitka's (2002) sample. One explanation for the high endorsement of RTI is that treatment validity/RTI has gained considerable notoriety of late. This can be seen in journal articles (e.g., L. S. Fuchs & Fuchs, 1998; L. S. Fuchs, Fuchs, & Speece, 2002) and conferences (for a comprehensive overview of one symposium, see the National Research Center on Learning Disabilities Response–to–Intervention Symposium, Kansas City, MO http://www.nrcld.org/symposium2003/index.html). It has also been endorsed or contemplated by many agencies in the special education field: D. Fuchs, Mock, Morgan, and Young (2003) listed eight national organizations that have recently discussed or advocated an RTI model for RD determination. In addition, efforts to implement such models have been featured prominently in the required reading for many school psychology training programs. For example, the first three chapters (Deno, 2002; Reschly & Ysseldyke, 2002; Tilly, 2002) of Best Practices in School Psychology, Fourth Edition (NASP, 2002), are devoted to the “problem–solving model” which has been deemed synonymous with, and a precursor to, the RTI model for school psychologists (D. Fuchs, Fuchs, & Compton, 2004; D. Fuchs, Mock, Morgan & Young, 2003).

The fact that school psychologists self–described as “more knowledgeable” in assessing RD endorsed treatment validity/RTI significantly higher than “less knowledgeable” respondents also lends support to this interpretation as these professionals might be more current on information of new practices disseminated through conferences and field–specific literature. Further, it is likely that the recent IDEIA 2004 reauthorization had an impact. The addition of an explicit reference to RTI in IDEIA 2004, particularly coupled with explicit wording making the IQ–achievement discrepancy unnecessary, likely caught the attention of many school psychologists.

Of course, the RTI model must also be appealing in order for it to be endorsed. In addition to the benefits advocates of the model promise for students, it may be attractive to school psychologists because of the expected role changes it may entail. Experts have called for school psychologist role changes from one of predominantly assessor to that of problem–solver with academic domain–specific consultation skills (Bramlett, Murphy, Johnson, & Wallingsford, 2002). Researchers have also noted school psychologists’ continued desire to devote more professional efforts toward direct interventions and problem–solving consultation (Hosp & Reschly, 2002). Although not addressed directly in the current study, RTI probably appeals to school psychologists because of such desired role changes.

School psychologists endorsed cutoff scores on reading achievement at a lower rate than journal editors. Although a fair majority (66.5%) of school psychologists believed it was an appropriate criterion, significantly more (83.7%; p <. 05) of Speece and Shekitka's (2002) sample endorsed it. Because we felt that CBM, not subsumed within an RTI approach, may also be an endorsable option for RD assessment, we added this criterion to the list of possible operational options. By “not subsumed within an RTI approach”, we mean models which use CBM probes to identify discrepancies between a referred student and their peers, but are not necessarily conceptualized as “dual–discrepancy” models, which place more emphasis on multiple tiers and longitudinal analysis of both the child's achievement level and slope (rate) of reading compared to local peers. Such non–RTI models would likely use CBM data toward eligibility early in the identification process because they do not emphasize unresponsiveness as much as contemporary RTI approaches. Approximately 73% of school psychologists in the current sample endorsed this option. Research has shown that a CBM approach is highly accepted; however, a considerable proportion of school psychologists do not use this form of assessment in their daily practice (Eckert & Shapiro, 1995; Chafouleas, Riley–Tillman, & Eckert, 2003; Shapiro, Angello, & Eckert, 2004). In fact, Nelson and Machek (in press) found that over 60% of their sample of practicing school psychologists reported never or infrequently using CBM when assessing reading.

The endorsement of cognitive processes (77.6%) and the IQ–achievement discrepancy model (61.9%) represents the two items that differed the greatest (as well as significantly; p <. 05) from Speece and Shekitka's (2002) survey of experts. These results emphasize the use of traditional cognitive measures, and may seem somewhat paradoxical given the high endorsement of the RTI model, in which the role of IQ tests would seemingly be diminished (e.g., see Fletcher, Coulter, Reschly, & Vaughn, 2004). Further, RTI advocates often stress the inadequacies of IQ testing within a discrepancy model as a basis from which to compare the advantages of their model.

However, that practicing school psychologists would be distinguished from journal editors on these results makes sense given the historical and current emphasis on IQ testing in training and practice (Goh, Teslow, & Fuller, 1981; Hutton & Dubes, 1992; Stinnett, Havey, & Oehler–Stinnett, 1994; Wilson & Reschly, 1996). And, because a large–scale change to a problem–solving approach such as RTI would entail significant role changes and the development of new skills in order for school psychologists to remain employed (Reschly & Ysseldyke, 2002), it is not surprising that many practitioners would endorse a method in which they are specialized and, so, seemingly protected by their niche demand. In fact, a recent survey found that 30% of responding school psychologists thought that a decreased emphasis on IQ tests in RD identification would impact their job security (Machek & Nelson, 2007). Additionally, some (e.g., Hyman & Kaplinski, 1994) have argued in the school psychology literature that loss of school psychologists’ assessment roles would likely jeopardize the profession. So, while there appears to be a willingness by many school psychologists to embrace new forms of service delivery, such as those offered by a RTI approach, it may be tempered by an apprehension about undetermined new roles and even job security fears if established roles are abandoned.

Measurement of specific cognitive processes has been promoted by a number of professionals, as mentioned above. For example, proponents of cross–battery assessment have argued that deficits in certain Cattell–Horn–Carroll Narrow Abilities are related to learning problems (e.g., Flanagan, Ortiz, Alfonso, & Mascolo, 2002). Specific to RD, Semrud–Clikeman (2003) reported the usefulness of assessing, among other neuropsychological processes, speed of information processing and working memory.

The difference between journal editors and practicing school psychologists on the use of an IQ–achievement discrepancy approach, or cognitive tests in general, may simply be a matter of practicality. It is possible that school psychologists see a practical need for such a formula while “changing over” to a new, largely untested, system of RD identification. Given the professional demographics of the Speece and Shekitka (2002) sample, perhaps those journal editors were able to base their responses more in regard to research than with an eye toward immediate practical application.

Further, it may be that many school psychologists see the value in both an RTI process, as well as one that looks at cognitive factors and/or severe discrepancies between cognitive measures and measures on standardized achievement tests. The coexistence of cognitive testing and an RTI model has been put forth by other researchers. D. Fuchs, Mock, Morgan, and Young (2003) envisioned a two–level, standard–treatment RTI protocol approach for RD identification. In addition to assessing response to instruction, their approach would make use of cognitive assessment procedures for all children at the second level, thus ruling out mental retardation, IQ–commensurate reading performance, as well as inadequate instruction. Paramount in their reasoning is their desire to retain the historical conceptualization of RD as unexpected underachievement. This desire has been voiced by others and unexpected underachievement has been conceptualized as a necessary but not sufficient criterion for distinguishing RD (Kavale & Forness, 2000). Further, there is a growing consensus on one side of the RTI debate that a truly comprehensive evaluation would need to include a number of valid procedures, and that, “RtI and cognitive assessment should be intimately entwined in SLD assessment to best serve children.” (Kavale, Kaufman, Naglieri, & Hale, 2005, p. 22).

The results of the current study suggest that the previous quote captures the sentiment of many practicing school psychologists. Given the continued polarized atmosphere surrounding RTI, this may be an affront to some. However, these results are important to the LD field as a whole given the prominent role that school psychologists have in LD determinations in many areas. Any further move in the direction of an RTI process for future identification will bring with it increased need for role expansion, role clarification, and increased collaboration among all of those involved in an RTI process. Amidst this collaboration, preferred practices and existing beliefs about the most valid and feasible components of identification practices will need to be debated and refined into practice. The current study presents a portrait of current best practice beliefs among school psychologists.

Consistent with federal guidelines, school psychologists overwhelmingly preferred consideration of exclusionary criteria when assessing RD. Our sample endorsed mental retardation and inadequate instruction most highly, which is consistent with what journal editors endorsed. These two criteria evoke both an RTI approach (which holds the promise of ruling out inadequate instruction) and reliability of measures of IQ (for ruling out mental retardation). Thus, these results again are consistent with the “dual” endorsement of a RTI approach, as well as one that specifically address general cognitive functioning.

Limitations

First, only NASP members were surveyed. Though most (70%) school psychologists are members of NASP (Fagan, 1994), members and non–members may differ on some characteristics (e.g., job satisfaction; Levinson, Fetchkan, & Hohenshil, 1988). A sparse amount of research has been conducted comparing NASP and non–NASP members, and, thus, very little specific information is known about potential differences between these groups. Therefore, the results of this study should not be generalized to those who are not members of NASP.

Secondly, although our response was consistent with those from other survey studies of school psychologists, the return rate of 37% does limit generalizability. The sample's shared characteristics with NASP member demographic information provides some justification for generalizing the results to NASP membership at large, but it should be highlighted that some uncontrolled variable(s), uncharacteristic of the average member, might have been operative in determining who did and did not respond to the survey. Specifically, it may be that mostly those with particularly strong attitudes on the subject matter responded. Following–up with non–responders to assess their opinions would have strengthened our results. The fact that the two surveys were conducted at two different points in time, particularly before and after significant changes in federal regulations, necessarily makes direct comparisons between them more problematic.

Finally, we added an additional item regarding the use of curriculum–based measures of reading assessment, and also changed the word “criterion” to “cutoff”, based on our piloting feedback. Any changes between surveys holds the possibility of reducing the validity of comparison(s).

Conclusion

Speece and Shekitka's (2002) interpretation of their results noted the RD field's difficulty in letting loose the IQ–achievement discrepancy component. They cited the need to look at RTI as well as “individual differences” (p. 122) in making special education designations. The authors summarized their results as meaning journal editors would emphasize an RTI approach, have an eye toward phonological processing deficits, and take care to rule out mental retardation as an etiological factor in reading difficulties. The school psychologists in our study also extolled the virtues of newly proposed alternative approaches to RD identification (e.g., treatment validity/RTI and measurement of phonological processing skills). At the same time, they supported the use of intelligence measures in diagnosing RD, as well as in ruling out mental retardation as an etiological factor. Though there are many who would like to see the use of IQ testing significantly reduced in RD classification (Gresham et al., 2005), most practicing school psychologists in our study did not seem to share this view.

As debate heats up regarding the use of intelligence measures and the primacy of the RTI approach in RD classification, any state or LEA efforts to transform RD assessment procedures will need to be planful. Considerations regarding existing skill bases and opinions of currently practicing school psychologists will have to be considered. Particular care will need to be given to possible changes in the school psychologist's role within any new RD identification framework. Since IDEIA 2004 leaves the door open for states and LEAs to develop their own RD classification standards, factions espousing a particular method will need to continue to establish credible evidence of its benefits and be able to promote such benefits to all those involved in the diagnosis and remediation of reading difficulties. Understanding the current perceptions of those stakeholders will be important when addressing further attitude change. The current study elaborates on the perceptions of school psychologists, a potentially important population of professionals assumed to be critical components within up current and future RTI implementation efforts.

Acknowledgments

Data collection efforts for this article were funded by a Minot State University Small Grant.

We would like to thank Deborah Speece for her willingness to share her original survey content and data. However, all opinions and statements contained herein are those of the authors.

Gratitude is extended to Caleb Case for his efforts in managing the data gathering and entry process for this study.

Footnotes

1

These criteria were expanded from the 3 possible choices (extremely knowledgeable, knowledgeable, not knowledgeable) used in the original study.

Appendices

Greg R. Machek, PhD, NCSP, is an Assistant Professor in the Department of Psychology at the University of Montana. He is a member of the School Psychology program faculty and has research interests in the role of school psychologists in identifying reading difficulties, bullying in the schools, and childhood psychopathology.

Jason M. Nelson, PhD, is an Assistant Professor in the Department of Psychology at the University of Montana. His research interests include the prevention of reading disabilities, role of the school psychologist in preventing reading disabilities, affective and motivational characteristics of students with reading disabilities, and internalizing psychopathology in children.

References

Aaron

P. G.

(1995). Differential diagnosis of reading disabilities. School Psychology Review, 24, 345–360.

Adams

M. J.

(1990). Beginning to read: Thinking and learning about print. Cambridge, MA : MIT Press.

Bramlett

R. K.

, Murphy

J. J.

, Johnson

, Wallingsford

, & Hall

J. D.

(2002). Contemporary practices in school psychology: A national survey of roles and referral problems. Psychology in the Schools, 39, 327–335.

Case

L. P.

, Speece

D. L.

, & Molloy

D. E.

(2003). The validity of a response–to–intervention paradigm to identify reading disabilities: A longitudinal analysis of individual differences and contextual factors. School Psychology Review, 32, 557–582.

Chafouleas

S. M.

, Riley–Tillman

, & Eckert

T. L.

(2003). A comparison of school psychologists’ acceptability, training, and use of norm–referenced, curriculum–based, and brief experimental analysis methods to assess reading. School Psychology Review, 32, 272–281.

Chafouleas

S. M.

, Clonan

S. M.

, & Vanauken

T. L.

(2002). A national survey of current supervision and evaluation practices of school psychologists. Psychology in the Schools, 39, 317–325.

Cohen

(1988). Statistical power analysis for the behavior sciences (2nd ed.). Hillsdale, NJ : Erlbaum.

Costenbader

V. K.

, Swartz

, & Petrix

(1992). Consultation in the schools: The relationship between preservice training, perception of conultative skills, and actual time spent in consultation. School Psychology Review, 21, 95–108.

Curtis

M. J.

, Lopez

A. D.

, Batsche

G. M.

, & Smith

J. C.

(March, 2006). School psychology 2005: A national perspective. Paper presented at the annual meeting of the National Association of School Psychologists, Anaheim, CA.

10.

Demaray

M. K.

, Schaefer

, & Delong

L. K.

(2003). Attention–deficit/hyperactivity disorder (ADHD): A national survey of training and current assessment practices in the schools. Psychology in the Schools, 40, 583–598.

11.

Deno

S. L.

(2002). Problem solving as best practice. In Thomas

& Grimes

(Eds.), Best practices in school psychology IV (pp. 37–55). Bethesda, MD : National Association of School Psychologists.

12.

Dombrowski

S. C.

, Kamphaus

R. W.

, & Reynolds

C. R.

(2004). After the demise of the discrepancy: Proposed learning disabilities diagnostic criteria. Professional Psychology: Research and Practice, 25, 364–372.

13.

Eckert

T. L.

, & Shapiro

E. S.

(1995). Teachers’ rating of the acceptability of curriculum–based assessment methods. School Psychology Review, 24, 497–508.

14.

Fagan

T. K.

(1994). A critical appraisal of the NASP's first 25 years. School Psychology Review, 23, 604–618.

15.

Flanagan

D. P.

, Ortiz

S. O.

, Alfonso

V. C.

, & Mascolo

J. T.

(2002). The achievement test desk reference (ATDR): Comprehensive assessment and learning disabilities. Boston : Allyn & Bacon.

16.

Fletcher

J. M.

, Coulter

W. A.

, Reschly

D. J.

, & Vaughn

(2004). Alternative approaches to the definition and identification of learning disabilities: Some questions and answers. Annals of Dyslexia, 54, 304–331.

17.

Fletcher

J. M.

, & Denton

(2003, December). Validity of alternative approaches to the identification of LD: Operationalizing unexpected underachievement. Paper presented at the National Research Center on Learning Disabilities Responsiveness–to–Intervention Symposium, Kansas City, MO.

18.

Fletcher

J. M.

, Francis

D. J.

, Shaywitz

S. E.

, Lyon

G. R.

, Foorman

B. R.

, Stuebing

K. K.

, & Shaywitz

B. A.

(1998). Intelligence testing and the discrepancy model for children with learning disabilities. Learning Disabilities Research & Practice, 13, 186–203.

19.

Fletcher

J. M.

, Shaywitz

S. E.

, Shankweiler

D. P.

, Katz

, Liberman

I. Y.

, Steubing

K. K.

, . (1994). Cognitive profiles of reading disability: Comparisons of discrepancy and low achievement definitions. Journal of Educational Psychology, 86, 6–23.

20.

Fuchs

L. S.

, & Fuchs

(1998). Treatment validity: A unifying concept for reconceptualizing the identification of learning disabilities. Learning Disabilities Research & Practice, 13, 204–219.

21.

Fuchs

, Mock

, Morgan

P. L.

, & Young

C. L.

(2003). Responsiveness–to intervention: Definitions, evidence, and implications for the learning disabilities construct. Learning Disabilities Research & Practice, 18, 157–171.

22.

Fuchs

L. S.

, Fuchs

, & Speece

D. L.

(2002). Treatment validity as a unifying construct for identifying learning disabilities. Learning Disabilities Quarterly, 25, 33–45.

23.

Fuchs

, Fuchs

L. S.

, & Compton

D. L.

(2004). Identifying reading disabilities by responsiveness to intervention: Specifying measures and criteria. Learning Disabilities Quarterly, 27, 216–227.

24.

Goh

D. S.

, Teslow

C. J.

, & Fuller

G. B.

(1981). The practice of psychological assessment among school psychologists. Professional Psychology, 12, 696–706.

25.

Good

R. H.

, Creek

R. J.

, Vollmer

, & Chowdhri

(1993). Treatment utility of the Kaufman Assessment Battery for Children: Effects of matching instruction and student processing strength. School Psychology Review, 22, 8–27.

26.

Gresham

F. M.

(2002). Responsiveness to intervention: An alternative approach to the identification of learning disabilities. In Bradley

, Danielson

, & Hallahan

D. P.

(Eds.), Identification of learning disabilities: Research to practice (pp. 467–564). Mahwah, NJ : Erlbaum.

27.

Gresham

F. M.

, Reschly

D. J.

, Tilly

W. D.

, Fletcher

, Burns

, Christ

, . (2005). Comprehensive evaluation of learning disabilities: A response to intervention perspective. The School Psychologist, 59, 26–30.

28.

Gresham

F. M.

, & Witt

J. C.

(1997). Utility of intelligence testing for treatment planning, classification, and placement decisions: Recent empirical findings and future directions. School Psychology Quarterly, 12, 249–267.

29.

Hosp

J. L.

, & Reschly

D. J.

(2002). Regional differences in school psychology practice. School Psychology Review, 31, 11–29.

30.

Hutton

J. B.

, & Dubes

(1992). Assessment practices of school psychologists: Ten years later. School Psychology Review, 21, 271–285.

31.

Hyman

I. A.

, & Kaplinski

(1994). Will the real school psychologist please stand up: Is the past a prologue for the future of school psychology? School Psychology Review, 23, 564–583.

32.

Individuals with Disabilities Education Improvement Act of 2004 (IDEIA), Public L. No.108–446, § 614 (2004).

33.

Kavale

K. A.

, & Forness

S. R.

(2000). What definitions of learning disability say and don't say: A critical analysis. Journal of Learning Disabilities, 33, 239–256.

34.

Kavale

K. A.

, Fuchs

, & Scruggs

(1994). Setting the record straight on reading disability and low achievement: Implications for policy making. Learning Disabilities Research & Practice, 9, 70–77.

35.

Kavale

K. A.

, Kaufman

A. S.

, Naglieri

J. A.

, & Hale

J. B.

(2005). Changing procedures for identifying learning disabilities: The danger of poorly supported ideas. The School Psychologist, 59, 16–25.

36.

Lerner

(1989). Educational intervention in learning disabilities. Journal of the American Academy of Child and Adolescent Psychiatry, 28, 326–331.

37.

Levinson

E. M.

, Fetchkan

, & Hehenshil

T. H.

(1988). Job satisfaction among practicing school psychologists revisited. School Psychology Review, 17, 101–112.

38.

Lyon

G. R.

, & Fletcher

J. M.

(2001, Summer). Early warning system. Education Next. Retrieved September 2, 2005, from http://www.educationnext.org/20012/22.html

39.

Machek

G. R.

, & Nelson

J. M.

(2007). School psychologists’ perceptions of past and proposed practices in reading disability identification. Manuscript in preparation.

40.

Mellard

D. F.

, Deshler

D. D.

, & Barth

(2004). LD identification: It's not simply a matter of building a better mousetrap. Learning Disabilities Quarterly, 27, 229–242.

41.

Naglieri

J. A.

(2005, March). IDEIA, SLD, RTI and cognitive assessment. Paper presented at the annual meeting of the National Association of School Psychologists, Atlanta, GA.

42.

Naglieri

J. A.

, & Das

J. P.

(1997). Cognitive Assessment System. Itasca, IL : Riverside.

43.

National Reading Panel. (2000). Teaching children to read: An evidence–based assessment of the scientific research literature on reading and its implications for reading instruction. Washington, DC : National Institute of Child Health and Human Development.

44.

Nelson

J. M.

, & Machek

G. R.

(in press). National survey of school psychologists’ training, practice, and perceptions of competence in reading assessment and intervention: An update and expansion. School Psychology Review.

45.

Nickerson

A. B.

, & Zhe

E. J.

(2004). Crisis prevention and intervention: A survey of school psychologists. Psychology in the Schools, 41, 777–788.

46.

Reschly

D. J.

(1997). Utility of individual ability measures and public policy choices for the 21st century. School Psychology Review, 26, 234–242.

47.

Reschly

D. J.

(2003, December). What if LD identification changed to reflect research findings? Paper presented at the National Research Center on Learning Disabilities Responsiveness–to–Intervention Symposium, Kansas City, MO.

48.

Reschly

D. J.

, & Hosp

J. L.

(2004). State SLD identification policies and practices. Learning Disabilities Quarterly, 27, 197–213.

49.

Reschly

D. J.

, & Wilson

M. S.

(1995). School psychology faculty and practitioners: 1986 to 1991 trends in demographic characteristics, roles, satisfaction, and system reform. School Psychology Review, 24, 62–80.

50.

Reschly

D. J.

, & Ysseldyke

J. E.

(2002). Paradigm shift: The past is not the future. In Thomas

and Grimes

(Eds.), Best practices in school psychology IV (pp. 3–20). Bethesda, MD : National Association of School Psychologists.

51.

Rutter

, & Yule

(1975). The concept of specific reading retardation. Journal of Child Psychology and Psychiatry, 16, 181–197.

52.

Scruggs

T. E.

, & Mastropieri

M. A.

(2002). On babies and bathwater: Addressing the problems of identification of learning disabilities. Learning Disabilities Quarterly, 25, 155–168.

53.

Semrud–Clikeman

(2003, December). Neuropsychological aspects for evaluating learning disabilities. Paper presented at the National Research Center on Learning Disabilities Responsiveness–to–Intervention Symposium, Kansas City, MMO.

54.

Shapiro

E. S.

, Angello

L. M.

, & Eckert

T. L.

(2004). Has curriculum–based assessment become a staple of school psychology practice? An update and extension of knowledge, use, and attitudes from 1990 to 2000. School Psychology Review, 33, 249–257.

55.

Siegel

L. S.

(1989). IQ is irrelevant to the definition of learning disabilities. Journal of Learning Disabilities, 25, 618–629.

56.

Speece

D. L.

, & Shekitka

(2002). How should reading disabilities be operationalized? A survey of experts. Learning Disabilities Research & Practice, 17, 118–123.

57.

Stanovich

K. E.

(1988). Explaining the differences between the dyslexic and the garden variety poor reader: The phonological–core variable–difference model. Journal of Learning Disabilities, 21, 590–612.

58.

Stinnett

T. A.

, Havey

J. M.

, & Oehler–Stinnett

(1994). Current test usage by practicing school psychologists: A national survey. Journal of Psychoeducational Assessment, 12, 331–350.

59.

Stuebing

K. K.

, Fletcher

J. M.

, LeDoux

J. M.

, Lyon

G. R.

, Shaywitz

S. E.

, & Shaywitz

B. A.

(2002). Validity of the IQ–discrepancy classifications of reading difficulties: A meta–analysis. American Educational Research Journal, 39, 469–518.

60.

Stanovich

K. E.

(1991). Discrepancy definitions of reading disability: Has intelligence led us astray? Reading Research Quarterly, 26, 1–29.

61.

Tilly

W. D.

(2002). Best practices in school psychology as a problem–solving enterprise. In Thomas

& Grimes

(Eds.), Best practices in school psychology IV (pp. 21–36). Bethesda, MD : National Association of School Psychologists.

62.

Torgesen

J. K.

(2002). The prevention of reading disabilities. Journal of School Psychology, 40, 7–26.

63.

Torgeson

J. K.

, & Wagner

R. K.

(1998). Alternative diagnostic approaches for specific developmental reading disabilities. Learning Disabilities Research & Practice, 13, 220–232.

64.

Wilczynski

S. M.

, Mandal

R. L.

, & Fusilier

(2000). Bridges and barriers to behavioral consultation. Psychology in the Schools, 37, 495–504.

65.

Wilson

M. S.

, & Reschly

D. J.

(1996). Assessment in school psychology training and practice. School Psychology Review, 25, 9–23.

66.

Ysseldyke

, Algozzine

, Shinn , & McGue

(1982). Similarities and differences between low achievers and students classified as learning disabled. The Journal of Special Education, 16, 73–85.