Understanding Differences in Instructional Quality between High and Lower Value-Added Schools in a Large Urban District

Abstract

Background/Context

High schools are under increasing pressure to move beyond just graduating students, and many high schools today continue to have low rates of student retention and learning, particularly for students from traditionally low-performing subgroups. Differential dropout rates, wherein low-income students, minorities, and English language learners leave school at higher rates than other students, only compound the problem.

Purpose/Objective/Research Question/Focus of Study

In this study, we examine differences in instructional quality between two higher and two lower value-added high schools, as measured by the Classroom Assessment Scoring System – Secondary (CLASS-S). It explores (1) differences in levels of instructional quality, (2) differences in the proportions of students taking advanced courses, and (3) differences in the way teachers think and talk about their classroom challenges.

Research Design

This is a mixed methods study that combines data from classroom observation protocols with teacher interview data. We use multilevel statistical models to address the first two research questions and emergent, inductive coding to determine commonalities within schools in how teachers implement higher-quality instructional practices.

Findings/Results

We find that the average difference in instructional quality, as measured by the CLASS-S, was not very wide across our four case study schools and that the biggest differences were between the two higher value-added high schools. Our interview data suggest that teachers in the two higher value-added schools are more proactive about providing emotional support and preventing behavioral problems, and intentional about attending to content and engaging students in higher-order thinking.

Conclusions/Recommendations

The lack of variation in classroom instructional practice across schools also suggests the need to attend to the ways that schools support academic learning outside of the classroom. Qualitative findings signal that the quality of classroom instruction is not the only critical input to students’ learning gains when trying to identify what leads schools to place highly in value-added rankings.

References

Allensworth

, Correa

, & Ponisciak

(2008). From high school to the future: ACT preparation—too much, too late. Consortium on Chicago School Research (CCSR). Retrieved from http://eric.ed.gov/?id=ED501457 on July 2, 2008.

Alper

, Fendel

, Fraser

, & Resek

(1997). Designing a high school mathematics curriculum for all students. American Journal of Education, 106(1), 148–178.

Anderman

, Andrzejewski

, & Allen

(2011). How do teachers support students' motivation and learning in their classrooms? Teachers College Record, 113(5), 969–1003.

Applebee

A. N.

, Langer

J. A.

, Nystrand

, & Gamoran

(2003). Discussion-based approaches to developing understanding: Classroom instruction and student performance in middle and high school English. American Educational Research Journal, 40(3), 685–730.

Bazeley

(2007). Qualitative data analysis with NVivo. London, England: Sage Publications Ltd.

Becker

B. E.

, & Luthar

S. S.

(2002). Social-emotional factors affecting achievement outcomes among disadvantaged students: Closing the achievement gap. Educational Psychologist, 37(4), 197–214.

Bell

C. A.

, Gitomer

D. H.

, McCaffrey

, Hamre

, & Pianta

R. C.

(April 2011). An argument approach to observation protocol validity. Paper presented at the annual conference of the American Education Research Association, New Orleans, LA.

Berliner

(1988). Implications of studies on expertise in pedagogy for teacher education and evaluation. In New directions for teacher assessment (Proceedings of the 1988 ETS Invitational Conference, pp. 39–68). Princeton, NJ: Educational Testing Service.

Boaler

, & Staples

(2008). Creating mathematical futures through an equitable teaching approach: The case of Railside School. Teachers College Record, 110(3), 608–645.

10.

Brown

(2008). Assessment and academic identity. Teachers College Record, 110(10), 2116–2147.

11.

Charmaz

(2006). Construction grounded theory: A practical guide through qualitative analysis (1st ed.). London, England: Sage Publications Ltd.

12.

Clotfelter

C. T.

, Ladd

H. F.

, & Vigdor

J. L.

(2007). Teacher credentials and student achievement: Longitudinal analysis with student fixed effects. Economics of Education Review, 26(6), 673–682.

13.

Cohen

, & Hill

(2000). Instructional policy and classroom performance: The mathematics reform in California. Teachers College Record, 102(2), 294–343.

14.

Cohen-Vogel

, & Harrison

(2013). Leading with data: Evidence from the National Center on Scaling Up Effective Schools. Leadership and Policy in Schools, 12(2), 122–145.

15.

Cook

, & Evans

W. N.

(2000, October). Families or schools? Explaining the convergence in white and black academic performance. Journal of Labor Economics, 18, 729–754.

16.

Cothran

, Kulinna

, & Garrahy

(2003). “This is kind of giving a secret away …”: Students' perspectives on effective class management. Teaching and Teacher Education, 19(4), 435–444.

17.

Curby

T. W.

, LoCasale-Crouch

, Konold

T. R.

, Pianta

R. C.

, Howes

, Burchinal

, … Barbarin

(2009). The relations of observed pre-K classroom quality profiles to children's achievement and social competence. Early Education and Development, 20(2), 346–372.

18.

Darling-Hammond

, Ancess

, & Ort

S. W.

(2002). Reinventing high school: Outcomes of the coalition campus schools project. American Educational Research Journal, 39(3), 639–673.

19.

Edhlund

(2011). NVivo 9 essentials. Stallarholmen, Swden: Form & Kunskap AB.

20.

Emmer

E. T.

, & Stough

L. M.

(2001). Classroom management: A critical part of educational psychology, with implications for teacher education. Educational Psychologist, 36(2), 103–112.

21.

Firestone

, & Herriott

(1982). Prescriptions for effective elementary schools don't fit secondary schools. Educational Leadership, 40(3), 51–53.

22.

Gamoran

, Nystrand

, Berends

, & LePore

P. C.

(1995). An organizational analysis of the effects of ability grouping. American Educational Research Journal, 32(4), 687–715.

23.

Glasser

, & Strauss

(1967). Discovery of grounded theory: Strategies for qualitative research. Chicago, IL: Adeline.

24.

Goldring

, Porter

, Murphy

, Elliott

S. N.

, & Cravens

(2009). Assessing learning-centered leadership: Connections to research, professional standards, and current practices. Leadership and Policy in Schools, 8(1), 1–36.

25.

Gregory

, & Ripski

(2008). Adolescent trust in teachers: Implications for behavior in the high school classroom. School Psychology Review, 37(3), 337–353.

26.

Grissom

J. A.

, Kalogrides

, & Loeb

(2014). Using student test scores to measure principal performance. Educational Evaluation and Policy Analysis, 37(1), 3–28.

27.

Harris

D. N.

(2011). Value-added measures in education. Cambridge, MA: Harvard Education Press.

28.

Harris

D. N.

, & Sass

T. R.

(2011). Teacher training, teacher quality and student achievement. Journal of Public Economics, 95(7), 798–812.

29.

Hill

H. C.

, Blunk

, Charalambous

, Lewis

, Phelps

G. C.

, Sleep

, & Ball

D. L.

(2008). Mathematical knowledge for teaching and the mathematical quality of instruction: An exploratory study. Cognition and Instruction, 26, 430–511.

30.

Hill

H. C.

, Kapitula

, & Umland

(2011). A validity argument approach to evaluating teacher value-added scores. American Educational Research Journal, 48(3), 794–831.

31.

Kagan

D. M.

(1992). Professional growth among preservice and beginning teachers. Review of Educational Research, 62(2), 129–169.

32.

Kane

T. J.

, Taylor

E. S.

, Tyler

J. H.

, & Wooten

A. L.

(2010). Identifying effective classroom practices using student achievement data (No. w15803). New York, NY: National Bureau of Economic Research.

33.

Kaufman

, & Chapman

(2004). Dropout rates in the United States: 2001 (NCES 2004–057), Table A-1. Data from U.S. Department of Commerce, Bureau of the Census, Current Population Survey (CPS), October Supplement, 1972–2001.

34.

Klem

, & Connell

(2004). Relationships matter: Linking teacher support to student engagement and achievement. Journal of School Health, 74(7), 262–273.

35.

LaParo

K. M.

, Pianta

R. C.

, & Stuhlman

(2004). The classroom assessment scoring system: Findings from the prekindergarten year. The Elementary School Journal, 104(5), 409–426.

36.

Lee

(2002). Racial and ethnic achievement gap trends: Reversing the progress toward equity. Educational Researcher, 31(1), 3–12.

37.

Lee

(2004). Multiple facets of inequity in racial and ethnic achievement gaps. Peabody Journal of Education, 79(2), 51–73.

38.

Marzano

R. J.

(2001). Designing a new taxonomy of educational objectives. Thousand Oaks, CA: Corwin Press.

39.

Measures of Effective Teaching Project. (2012). Gathering feedback for teaching: Combining high-quality observations with student surveys and achievement gains. Bill & Melinda Gates Foundation.

40.

Measures of Effective Teaching Project. (2013). Ensuring fair and reliable measures of effective teaching: Culminating findings from the MET Project's three-year study. Bill & Melinda Gates Foundation.

41.

Meyer

(1997). Value-added indicators of school performance: A primer. Economics of Education Review, 16(3), 283–301.

42.

Muller

, Katz

S. R.

, & Dance

L. J.

(1999). Investing in teaching and learning dynamics of the teacher-student relationship from each actor's perspective. Urban Education, 34(3), 292–337.

43.

Nye

, Konstantopoulos

, & Hedges

L. V.

(2004). How large are teacher effects? Educational Evaluation and Policy Analysis, 26, 237–257.

44.

Oakes

(2005). Keeping track: How schools structure inequality, Second Edition. New Haven, CT: Yale University Press.

45.

Patton

M. Q.

(2001). Qualitative research & evaluation methods (3rd ed.). St. Paul, MN: Sage.

46.

Pianta

R. C.

, Hamre

B. K.

, Haynes

N. J.

, Mintz

S. L.

, & LaParo

K. M.

(2007). Classroom assessment scoring system – Secondary Manual. Charlottesville, VA: University of Virginia.

47.

Preston

, Goldring

, Guthrie

J. E.

, & Ramsey

(2012, June). Conceptualizing essential components of effective high schools. Paper presented at the Achieving Success at Scale: Research on Effective High Schools conference, Nashville, TN.

48.

QSR International Pty Ltd. (2008). NVivo Qualitative data analysis. Melbourne, Australia: QSR International Pty Ltd.

49.

Rampey

B. D.

, Dion

G. S.

, & Donahue

P. L.

(2009). NAEP 2008 Trends in Academic Progress (NCES 2009–479). National Center for Education Statistics, Institute of Education Sciences, U.S. Department of Education, Washington, DC.

50.

Raudenbush

S. W.

, & Bryk

A. S.

(2002). Hierarchical linear models: Applications and data analysis methods (2nd ed.; Advanced Quantitative Techniques in the Social Sciences Series, No. 1). Thousand Oaks, CA: Sage.

51.

Rivkin

S. G.

, Hanushek

E. A.

, & Kain

J. J.

(2005). Teachers, schools, and academic achievement. Econometrica, 73(2), 417–458.

52.

Rockoff

J. E.

(2004). The impact of individual teachers on student achievement: Evidence from panel data. American Economic Review, 94(2), 247–252.

53.

Rockoff

J. E.

, Jacob

B. A.

, Kane

T. J.

, & Staiger

D. O.

(2008). Can you recognize an effective teacher when you recruit one? (NBER Working Paper 14485). Retrieved from http://www.nber.org/papers/w14485.

54.

Rosiek

(2003). Emotional scaffolding: An exploration of the teacher knowledge at the intersection of student emotion and the subject matter. Journal of Teacher Education, 54(5), 399–412.

55.

Sass

(2012, June). Selecting high and low-performing high schools in Broward County, Florida for analysis and treatment. Paper presented at the Achieving Success at Scale: Research on Effective High Schools conference, Nashville, TN.

56.

Snyder

T. D.

, Dillow

S. A.

, & Hoffman

C. M.

(2009). Digest of education statistics 2008 (NCES 2009-020). Washington, DC: National Center for Education Statistics, Institute of Education Sciences, U.S. Department of Education.

57.

Staples

(2007). Supporting whole-class collaborative inquiry in a secondary mathematics classroom. Cognition and Instruction, 25(2), 161–217.

58.

Wenglinsky

(2002). The link between teacher classroom practices and student academic performance. Educational Policy Analysis Archives, 10(2) 1–30.

59.

Wenglinsky

(2004). The link between instructional practice and the racial gap in middle schools. Research on Middle Level Education, 28(1), 1–13.