Abstract
In the performance-driven culture of education today, though arguments regarding the definition of student success abound, few argue its importance. This issue is complicated with an additional dimension for those who are learning English as a second language. For those students who lack proficiency in the English language, academic content must be learned concurrently with the acquisition of the English language. The issue of language acquisition became part of the national education dialogue with the Supreme Court Decision Lau v. Nichols in 1974. It has continued to be a serious issue throughout the country and particularly in the southeast region of Texas. This article will examine how English-language learners (ELLs) are assessed. Specifically, it will focus on the relationship between the standardized academic exam administered to almost all students in Texas and the language acquisition exam for reading administered to almost all ELL students in the state. Furthermore, this article will examine the results of a regression analysis performed to determine the predictive value of using raw scores on the reading test for ELLs to predict raw scores on the standardized academic exam taken by those students at Grades 3, 5 and 8 in the same academic year.
Introduction
What is student success? What does it look like? Is student success the ability to teach children how to perform well on a standardized test? Is student success determined by graduation rates? If so, would a graduation rate of just fewer than 50% be the legacy of “successful” educators?
Student success is measured in many different ways. Some measure success by grades on report cards and test scores. Others consider a student successful when they are promoted to the next grade level. Many consider that a student is successful when they graduate from high school. The State of Texas (Texas Education Agency [TEA], 2008h) defines student success, in part, as a student’s ability to meet minimum standards in core curriculum areas as assessed on the Texas Assessment of Knowledge and Skills (TAKS). Regardless of the tool used to measure success, every student should have equal opportunities to achieve that success. In recent years, federal legislators passed the No Child Left Behind Act (NCLB) in an attempt to level the playing field for all students (United States Department of Education [USDE], 2001). The goal of the legislation was to ensure equal access to a high-quality education.
NCLB emphasizes test scores and accountability for schools and for students (Hursch, 2007). Students are required to pass a standardized test to graduate from high school. In some states, children are required to pass standardized tests to be promoted to the next grade level. School districts and individual campuses are rated according to how all students score on various core curriculum areas of the state’s standardized test. Districts and campuses are also accountable for the performance of various subpopulations. Student scores are grouped according to socioeconomic status, ethnicity, Limited English Proficiency (LEP) status, and eligibility for special education services. Many school districts have recognized that their students are struggling with the standardized tests and are committed to assisting their students, especially LEP students (Black, 2006). NCLB targets closing the achievement gaps of particular subgroups, notably ethnic groups, students of low socioeconomic status, students with disabilities, and LEP students (Darling-Hammond, 2007; Rentner et al., 2003).
Texas developed a statewide testing system in the 1980’s, several years before the official mandates of NCLB were enacted (Heilig & Darling-Hammond, 2008). Over the last 30 years, schools in Texas measured student achievement through a progression of increasingly rigorous tests beginning with the Texas Assessment of Basic Skills (TABS) and the Texas Assessment of Minimum Skills (TEAMS). The Texas Assessment of Academic Skills (TAAS) was administered to all third- through tenth-grade students from 1994-2002. The only students eligible for exemption from the TAAS test were recent immigrants identified as LEP or students served by Special Education.
Currently, students are assessed with the criterion-referenced TAKS test (TEA, 2008a). TAKS is based on the state’s required curriculum, the Texas Essential Knowledge and Skills (TEKS; Sclafani, 2001). The TEA rates campuses as exemplary, recognized, academically acceptable, or academically unacceptable based on overall student performance and the performance of ethnic and socioeconomic subgroups on TAKS reading, mathematics, writing, science, and social studies (Assaf, 2006; McNeil, Coppola, Radigan, & Heilig, 2008; TEA, 2008a). Schools are accountable for any subgroup with at least 30 students when those 30 students comprise at least 10% of the student population. If a subgroup includes 50 students, schools are accountable for the group regardless of the percentage of the total student population they represent.
Under the Student Success Initiative (SSI), students are required to pass the reading TAKS test in the third, fifth, and eighth grade to be promoted to the next grade level (TEA, 2008h). Students must also pass Math TAKS in the fifth and eighth grade to be promoted. To be eligible for a high school diploma, Texas students must pass all sections of the TAKS test (TEA, 2007). These sections include English language arts, mathematics, science, and social studies.
There are some differences between federal and state accountability requirements. In contrast to the state rating system, NCLB holds schools and districts accountable for making Adequate Yearly Progress (AYP) in only reading, math, and science. The requirement for LEP students to demonstrate progress in proficiency in English is also unique to the AYP requirement. (TEA, 2008b).
Despite NCLB’s goal of closing the achievement gap, scholars debate its effectiveness (Heilig et al., 2008; McNeil et al., 2008). Some assert that schools have found ways to circumvent the accountability system that negatively impact minority and low socioeconomic groups. These strategies include retention to raise TAKS passing rates in the subsequent grade level, strict enforcement of attendance requirements attached to high school credits, and placement in special education programs. Some fear that these policies and increasing accountability will further exacerbate the achievement gap by raising the drop-out rate among struggling students. For example, less that 50% of Latino students in the United States graduate from high school (Orfield, Losen, Wald, & Swanson, 2004). Such issues prompt Fuller, Wright, Gesicki, and Kang (2007) to question the validity of allowing states to develop their own standards of proficiency and means of assessing student achievement under NCLB. They give preference to data from the National Assessment of Education Progress (NAEP) which indicates “the Latino-White gap has continued to close . . . the one bright spot on the equity front” (p. 275). The Latino subgroup serves as the focus of our study.
There is a documented achievement gap between LEP students and their native English-speaking peers (Verdugo & Flores, 2007). By law and ethical standards, educators have a responsibility to bridge this gap. In the landmark 1974 Supreme Court Case Lau v. Nichols, a class action suit was brought by Chinese students in California (Valverde, 2004). The court ruled students who speak a language other than English and received no bilingual instruction or support were being denied civil rights. These rights were protected by the 1964 Civil Rights Act and the 14th Amendment’s equal protection clause. Lau v. Nichols ignited the process of building a legal framework for addressing the educational needs of the growing LEP population.
As the diversity of the nation’s schools increases, it is incumbent on educators to change the nature of their instructional practices to meet the needs of the diverse learners they serve (Olmedo, 2004). In some urban areas, schools are serving a student population with 90% ethnic minority or language minority. According to Lenski, Ehlers-Zavala, Daniel, and Sun-Irminger (2006), approximately 10% of the students schools serve in the United States are LEP. How do we know if we are adequately serving these students?
Relative to acquiring academic English skills, it is easier for an LEP student to acquire basic conversational English. For students to develop adequate command of academic English, they need to study the language in the context of the core curriculum for 4 to 7 years (Verdugo & Flores, 2007). Academic English is defined as having a level of proficiency with complex sentence structure and vocabulary so a student’s academic performance is not adversely affected by limited language skills. This is particularly important because a student’s ability to acquire academic English is highly correlated with that student’s ability to achieve academic success. Texas state law has attempted to address this concern (TEA, 2008i).
In addition to TAKS, schools are required to rate LEP students on their proficiency with English (TEA, 2008i). The Texas English Language Proficiency Assessment System (TELPAS) measures growth in the areas of reading, writing, speaking, and listening for these LEP students. All areas are based on teacher observation, except for reading skills which are assessed using an objective test beginning in Grade 2. After taking the TELPAS reading test every year, LEP students are assigned ratings which designate levels of reading proficiency in English. Those ratings progress from the most rudimentary to the most advanced: Beginner, Intermediate, Advanced, and Advanced High. A student with the designation of Beginner is an LEP student with very little reading capability in English whereas a student designated Advanced High is capable of reading on a level comparable with their native English-speaking peers. Ideally, LEP students will progress across the rating continuum as their reading skills in English improve. These assessments allow schools to track progress in the acquisition of English language skills over consecutive years. Texas schools have the authority to declare recent immigrants as lacking requisite academic English skills and thereby exempt these students from accountability to the same standards as their non-LEP peers for up to 3 years. However, at the end of the third year, no further exemptions are allowed. If Verdugo and Flores (2007) indicate it takes 4 to 7 years to achieve academic English proficiency, is it reasonable for the state to require an LEP to acquire this level of language proficiency in only 3 years?
Researchers also express concern over the consequential validity of the tests with English language learners (Verdugo & Flores, 2007). When testing LEP students, one must be cautious about assuming the results of the test are reflective of the students’ academic capabilities. Testing these students without making allowances for the influence of their limited English proficiency may yield inaccurate results.
The purpose of this study was to explore the relationship between the TELPAS reading test and the TAKS reading test. We examined the relationship between the two tests in Grades 3, 5, and 8 because of the impact of student scores on promotion eligibility. If LEP students are legally accountable to the same academic standards which apply to their non-LEP peers, is the state doing a good job of producing a valid tool for assessing these students? Is there a relationship between the TELPAS reading test and the TAKS reading test? More specifically, is there a correlation between students’ scores on the TELPAS reading test and their performance on the TAKS reading test in the same year? If so, what is the predictive value of TELPAS reading scores for determining scores on the TAKS reading test in the same year?
Our hypothesis was that there is a correlation between TELPAS reading scores and TAKS reading scores. Students who achieve higher scores on TELPAS reading should pass the TAKS test more often than students with lower TELPAS reading scores.
Method
Sample
The researchers collected samples from two large, suburban school districts in southeast Texas. The type of sampling method used was a convenience sample because researchers had access to the information from the two school districts. Both school districts together serve a population of approximately 60,000 students from prekindergarten through the twelfth grade. For this study, students were chosen from third, fifth, and eighth grades. Only students who took both the English version of the TAKS reading test and the TELPAS reading test were used. The only students who take TELPAS reading are students who are designated LEP. In one school district, approximately 1,295 students took both TELPAS and TAKS in the 2007-2008 school year. There were approximately 2,970 students from the other school district. Of the 4,265 students, about 2,000 took the Spanish version of the TAKS reading test and were not included in our study. No students who met the criteria were excluded. The sample consisted of 2,270 students. Table 1 describes the demographic information for these students. As the chart indicates, the majority of students in the sample are of Hispanic ethnicity and are classified as low socioeconomic status.
Demographics for Study Sample (N = 2,270)
Research Design
In this study, the researchers conducted nonexperimental research using archival data. All data collected was from the 2007-2008 school year. The researchers determined the nature of the relationship between the scores of the two tests using Pearson’s correlation coefficient (Salkind, 2008). Because a statistically significant correlation was found, the researchers performed a regression to determine concurrent criterion-related validity.
Operational Definitions
For the purpose of this study, several terms were operationally defined. Students are designated LEP if they score as non-English speaking (NES) on an oral language proficiency test (OLPT). According to the Texas Education Code (TEA, 1996), an OLPT must be administered if the parents indicate any language other than English in response to either of the following two questions on the home language survey: (a) What language is spoken in your home most of the time? and (b) What language does your child speak most of the time? Raw scores were used for both TELPAS reading and TAKS reading. All students who took TAKS reading in English and TELPAS reading in the represented districts were included in the study. Any students not meeting these criteria were excluded from our study.
Research Procedures/Data Collection
Prior to conducting research for this study, permission was obtained from each district’s Institutional Review Board (IRB) and the University of Houston-Clear Lake’s Committee for the Protection of Human Subjects (CPHS). The IRB chairs in the two districts provided a data disk which contained information for all students in the grade levels included in the study. Using research parameter tools built into the data disk, the researchers extracted the data for students who met the criteria for the sample group. Once the exclusive parameters were set, the researchers ran a report from the data disk which provided raw scores for the TAKS reading and the TELPAS reading tests for the identified students.
Instrumentation
TAKS reading
The TAKS reading test served as a measure of reading achievement for the purposes of this study (TEA, 2008c, 2008d, 2008e). This pencil/paper test is administered to Texas students in Grades 3 through 9 each year. The test is comprised of reading passages followed by multiple test questions that test four objectives in Grades 3 through 8: Basic Understanding, Knowledge of Literary Elements, Analysis Using a Variety of Strategies, and Analysis Using Critical Thinking Skills. Students must earn a scale score of 2,100 to be considered to have “met standard.” Although students who do not meet the passing standard in Grades 3, 5, and 8 are allowed to retake the TAKS reading test up to two more times, researchers only included raw scores from the first reading test administration.
The 2007 Technical Manual for the Texas Assessment System reports reliability for the TAKS reading test as measured using the Kuder–Richardson Formula (KR20) as follows: Grade 3—0.890, Grade 5—0.862, and Grade 8—0.884 (TEA, 2007). The manual argues for the content validity of the TAKS reading test based on the process of test development. A diverse group of current and former educators from across Texas and from other states are contracted to write test items that measure test objectives derived from the state curriculum. Texas educators (including classroom teachers, administrators, curriculum specialists, and education serve center staff), TEA staff, and professional test developers serve on committees to review test items for appropriate difficulty levels and for potential bias. Selected items are field tested and committees review the results before placing items in a test item bank. The manual also reports criterion-related validity based on the Grade Correlation Study (TEA, 2007). This study compared the rate of students who passed the TAKS tests with their passing rates on related courses these students had previously taken.
TELPAS reading
The TELPAS reading test will serve as the measure of LEP students’ reading proficiency in English for this study (TEA, 2008f). The test was first administered in 2008. It assesses reading skills through multiple-choice items that range in complexity from choosing the word that best completes a sentence to choosing the best answer to comprehension questions that follow reading passages. The TELPAS reading test differs from its predecessor, the Reading Proficiency Test in English (RPTE), by including a greater number of items at the advanced high level of language proficiency and by more extensively assessing a student’s ability to read academic English, particularly in a math or science context. The TELPAS reading test rates a student’s reading skills as beginning, intermediate, advanced, or advanced high. Six versions of the test target different grade levels: Grades 2, 3, 4-5, 6-7, 8-9, and 10-12. In 2008, TEA provided districts the choices of a pencil–paper or an online version of the test.
TEA does not currently provide reliability information regarding the TELPAS reading test since its first administration was in the spring of 2008. However, the 2007 Technical Digest asserts test content validity based on a rigorous process of test development similar to that of the TAKS reading test (TEA, 2007). For the development of the TELPAS reading, TEA solicited input from language acquisition specialists and psychometric experts. TEA field tested TELPAS reading test items in 2006 and 2007 before administering the test for the first time in 2008.
Test development
TEA contracted Pearson Educational Management (PEM) to provide support for the TELPAS reading test and the TAKS reading test (TEA, 2007). PEM secured subcontractors such as Educational Testing Services (ETS) to work with TEA staff and Texas educators to write items for the TAKS reading test. Another subcontractor, Questar Assessment, Inc., helped develop items for the TELPAS reading test. Texas educators served on committees to review test items for appropriate difficulty levels and for potential bias as described previously. TEA then field tested these items before including them in a test item bank. PEM assumed responsibility for printing, distributing, scoring, and reporting results for both tests.
Test administration
Classroom teachers and other certified school personnel administer the TAKS reading and the TELPAS reading tests (TEA, 2009). Noncertified personnel may administer the tests when they are under the supervision of a certified educator. Test administrators must sign the Oath of Test Security and Confidentiality before each administration to indicate they have read and understand the test administrator manual and have been trained in testing procedures. Students may take as much time as they need to complete the test, but they must finish the exam by the end of the day. Districts have the option to limit testing to the regular school day or to extend testing time to after-school hours. TEA makes special provisions and accommodations for students with special needs on both the TAKS reading and TELPAS reading test and provides a manual that explains such accommodations.
Data Analysis
There was a twofold purpose for this study. The researchers first would determine if there was a statistically significant correlation between the TELPAS reading test and the TAKS reading test. Statistical significance was set at p < .05. If the researchers determined there was a statistically significant relationship by using Pearson’s correlation coefficient, a regression analysis would be performed. This analysis would allow the researchers to determine the value of using a student’s TELPAS reading raw score to predict that student’s TAKS reading raw score.
Ethical Issues
By submitting to the review process for each district’s IRB and the University of Houston-Clear Lakes’ CPHS, the researchers took the first step toward the protection of the participants for the study. The researchers also took additional precautions to protect the identity of the individual students. First, a large sample from two districts minimized the risk of individuals being identified. Second, the researchers assigned a random numeric designation to individual sets of data used in the study. As the study involved archival data collected previously for another purpose, informed consent was not required.
Results
Data was gathered from 2,270 participants, with approximately 50% of those participants being male. Almost 96% of the sample was Hispanic in ethnicity. About 89% of our sample was listed as receiving free or reduced price lunches, which is higher than the overall socioeconomic status for the two school districts studied.
A correlation coefficient was computed between the TELPAS reading raw scores and the TAKS reading raw scores reported for students in the sample. A p value of less than .05 was required for significance. Researchers found a strong positive correlation coefficient of .609 for the sample. When researchers broke the sample into grade-level subgroups, strong positive correlations were found consistently among the groups. Table 2 reports these findings. The results suggest that if students perform well on the TELPAS reading test, they tend to perform well on the TAKS reading test regardless of the grade level.
Correlations by Grade Level (N = 2,270)
p < .05
Researchers conducted a linear regression analysis to determine the predictive value of using a student’s TELPAS reading raw score to predict that student’s TAKS reading raw score. The scatterplot for the two variables, as shown in Figure 1, indicates that the two variables are linearly related such that as the raw scores increase on the TELPAS reading test, the raw scores on the TAKS reading test also increase. The regression equation for predicting TAKS reading performance is,
Scatterplot between raw score for TELPAS reading and raw score for TAKS reading
The scores are significantly related at the p < .05 level. Accuracy in predicting the TAKS reading raw scores was strong. About 37% of the variance of the TAKS reading raw scores was accounted for by their linear relationship with the TELPAS reading raw scores.
Additionally, researchers conducted a linear regression analysis to evaluate the prediction of the TAKS reading scores from the TELPAS reading scores for each grade level represented in the sample. Figures 2, 3, and 4 show the scatterplots for the two variables at each grade level studied. These indicate the two variables are linearly related such that as the raw scores increase on the TELPAS reading test, the raw scores on the TAKS reading test also increase. The regression equations for predicting TAKS reading performance at Grades 3, 5, and 8, respectively, are
Scatterplot between raw score for TELPAS reading and raw score for TAKS reading at third grade
Scatterplot between raw score for TELPAS reading and raw score for TAKS reading at fifth grade
Scatterplot between raw score for TELPAS reading and raw score for TAKS reading at eighth grade
The scores are significantly related at the p < .05 level. Accuracy in predicting the TAKS reading raw scores was strong. Table 3 reports the amount of variance for the TAKS reading raw scores that can be accounted for by their linear relationship with the TELPAS reading raw scores at each grade level.
r2 by Grade Level (N = 2,270)
Discussion
Explanation of Results
As hypothesized, the study revealed a strong positive correlation between a student’s results on the TELPAS reading test and the TAKS reading test for the same student at Grades 3, 5, and 8. In other words, students who perform well on the TELPAS reading test tend to perform well on the TAKS reading test. Additionally, our hypothesis that TELPAS reading has value for predicting an individual’s TAKS reading score at Grades 3, 5, and 8 proved to be correct. The tests can be said to have concurrent criterion-related validity; it can be said the higher a student’s score on the TELPAS reading test, the higher that same student will score on the TAKS test during that year and at that grade level.
Of interest to the researchers were the differences in the strength of the relationship between the two tests across the three grade levels which were examined. Whereas the correlation for the two tests was positive and strong at each individual grade level, the strength of the correlation decreased between Grade 3 and Grade 5. Researchers noted a similar decline between Grade 5 and Grade 8. Consequently, the coefficient of determination, r2, decreased as student grade levels increased. These observations indicate that as students increase in grade level, the strength of the relationship between the two tests decreases as does the amount of variance accounted for in student performance on both tests.
The researchers believe this may be due to multiple factors. First, in the two districts represented in the sample, there is an expectation that more LEP students will take the English version of the TAKS reading test at the fifth-grade level than at the third-grade level. Furthermore, TEA does not provide a Spanish version in eighth grade. Therefore, one would expect the sample groups to contain students with a greater range of English proficiency as the higher grades. Second, our sample contains a proportionately smaller number of students in Grade 8 than in Grade 5 and in Grade 5 than in Grade 3. This is due to students meeting exit criteria for the bilingual program and thereby losing their LEP status as they advance in grade levels. Third, students served through special education could represent a larger subgroup as students increase in grade level. These students struggle to pass the TAKS test and to reach levels of English proficiency required to meet exit criteria. Fourth, the percentage of the sample represented by recent immigrants increases across the grade levels as students who are proficient in English meet exit criteria and lose LEP status. Finally, the increasing range of the difficulty of test items across grade levels may also account for a weaker relationship between the two tests in the upper grades.
Another point of interest was the difference in the regression line for fifth grade as compared to the other regression lines the researchers analyzed. Although students received comparatively lower TAKS reading scores than TELPAS reading scores at fifth grade, the slope of the regression line is steeper. This indicates that as a fifth-grade student makes gains in their TELPAS reading score, their TAKS reading score increases more quickly than a student at any of the other grade levels studied. The researchers believe this difference could be attributed to a realigning in instructional setting for fifth-grade students in one of the districts studied.
Limitations of the Study
There are a number of limitations for making generalizations with the results of this study. Because this study was conducted with data from one school year and from only two districts, the results of this study can only be generalized to schools in the southeast Houston area. As this study was conducted in 1 year, it cannot be assumed that these results will generalize between grade levels for a student. The participants used in this study were primarily of Hispanic ethnicity. Researchers could not assume they would find the same results when studying an LEP population with majority representation from another ethnic group. In addition, the results for students taking an alternative form of the TAKS reading test (i.e., TAKS Accommodated, TAKS Modified, and TAKS Alternative) were excluded from the sample. Therefore, results may only be generalized for students taking the regular TAKS reading test. Additionally, internal validity issues concern researchers because the data did not distinguish between results for students who took the online format of the TELPAS reading test and those who took the pencil and paper format. Therefore, no assumptions should be made specific to either format. These factors might be the focus of further studies.
Implications for Further Research
Expanding the scope of this study to include students across the state of Texas would reveal if statewide results are consistent with the findings in this study. Also, future studies may focus on a different demographic such as socioeconomic status or a different ethnicity to determine if the results are consistent across different subgroups. Perhaps researchers might include the alternative forms of the TAKS reading test taken by students served by special education programs. Any of these studies would enable researchers to generalize findings over a much broader population. Another variable worthy of attention would be the test format. Now that all LEP students take the online version of the TELPAS reading test with rare exception, replicating the study to determine if findings are consistent would be of interest.
The next step in the study would be to evaluate the predictive value of the TELPAS reading test scores in 1 year on the TAKS reading scores the following year. If a regression analysis proves significant, this information could be used to plan appropriate instructional programs for students between the two tests. Also, recent state-level legislative changes regarding the process by which special education students can be exited from LEP status would make replication of the study a worthy endeavor. School districts are now able to alter exit criteria for LEP students served in special education programs. In the next few years, as more of these students meet exit criteria, the population of students assessed by TELPAS could change, particularly at the fifth- and eighth-grade levels. The researchers believe this change could lead to a stronger correlation between the two tests at Grades 5 and 8 and, consequently, yield a more accurate predictive value of the TELPAS reading test scores for TAKS reading test scores. This would enable school leaders and teachers to provide more appropriate interventions for their LEP students.
Footnotes
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.