Twenty-first century adaptive teaching and individualized learning operationalized as specific blends of student-centered instructional events: A systematic review and meta-analysis

4.1.1 Types of studies

Only studies that considered the difference between the two groups were eligible for inclusion in this review. Included are studies that are experimental (i.e., Randomized Control Trials) or high-quality Quasi-Experimental Designs (i.e., statistically verified group equivalence or adjustment) in design that adequately addressed the research question of group comparisons, contained legitimate measures of academic achievement (i.e., teacher/researcher-made, standardized), and reported sufficient statistical information for effect size extraction.

4.1.2 Types of participants

The participants are students in k-12 formal educational settings (∼ages 5–18) eventually leading to a certificate, diploma, degree, or promotion to a higher level. Educational interventions take place either in the classroom, via distance education, or as a blended intervention (various combinations of classroom and distance education).

4.1.3 Types of interventions

As described earlier, the intervention in question (an experimental condition) was considered to be any combination of instructional events that is rated higher in Student-Centered qualities than a comparison (control) condition. Student participation in decisions about or control over the selection of study materials and learning activities, pacing of instruction, adapting learning for students’ individual needs, interests, backgrounds, etc., as well as various degrees of involvement in “partnership” with teachers, constitute, in our view, such Student-Centered qualities of instruction. Two experienced independent reviewers coded instructional conditions featured in a given primary study (on a scale from 1–5) to reflect the extent to which each group possessed these qualities. Below we describe dimensions that were in the focus of our review.

Within each eligible comparative study all participation groups were coded for the four effect dimensions using a five-point scale, as follows:

Dimension of Teacher's Role represents a continuum of a teacher's major responsibilities for organizing/delivering instruction/managing classroom activities, etc.

Coding: Describes the teacher's predominant role in the teaching/learning process:

1.

Teacher almost exclusively lectures, is the main source of content-relevant information and/or an authority figure.

2.

Teacher provides some guidance, feedback, initiates and supports discussions, etc.

3.

Teacher functions as a guide, coach, tutor, provocateur of thinking.

4.

Teacher functions as a colleague, partner in learning.

5.

Teacher almost exclusively acts as a facilitator of learning, responding to students’ specific needs (follows students’ lead, consults, clarifies, encourages, etc.).

Dimension of Pacing reflects the degree of student control over the time of instruction/learning and over the progression through the course content (i.e., pedagogical flexibility—revisiting/selecting/skipping/reordering topics and tasks).

Coding: Describes the degree to which students are given control over course progression: 1.

Instruction is highly structured and progresses step-by-step; no flexibility is allowed.

2.

Minor degree of either logistical or pedagogical flexibility is available to students.

3.

Program/teacher's control over course progression is balanced with that of students.

4.

Students have a substantial amount of flexibility in course progression.

5.

High degree of flexibility (up to the point of completely self-paced and/or self-planned/self-managed learning).

Dimension of Flexibility describes the degree of student control over course design, selection, and the provision of study materials and the setting up of learning objectives.

Coding: Describes the degree to which teachers/students participate in course planning: 1.

No involvement of students (most is determined by the teacher or program/curriculum).

2.

Student involvement in at least one of the components of course planning is present but limited.

3.

Teachers and students collaborate in the course planning, but teacher's role is still dominant.

4.

Teachers and students collaborate in the course planning equally.

5.

High student involvement—students play a leading role in course planning and selection of learning materials.

Dimension of Adaptability of Instruction describes the degree to which levels or modifications in instructional process is provided to accommodate individual students.

Coding: Describes the degree to which instruction takes into account students’ needs/interests/level of knowledge: 1.

Learning materials, settings, group formation (if any), activities and other work arrangements are predetermined and unchanged throughout the instruction (e.g., standardized or required curriculum).

2.

Minor modifications are allowed to either learning materials, group composition, or the context of instruction.

3.

Elements of either individualized feedback, or role and tasks assignments based on students’ interests and/or previous achievements, etc.

4.

Adapting several instructional components (in combinations) to students’ individual needs/interests/levels of knowledge.

5.

High levels of joint Adaptability of several components of instruction.

Based on the results of this coding (implemented independently by two reviewers compared and finalized in discussions), numeric values for each participating group were derived. The sum of these values determined the experimental (higher total) and control (lower total) conditions in every included study. The differential score was subsequently calculated to reflect the degree of student-centered (Student-Centered) components of instruction and to serve as a “continuous” predictor in meta-regression of effect sizes against the “strength” of the intervention. In the Results section this variable is depicted as “Student-Centered Total Differential Score” (i.e., sum of scores for the experimental group minus sum of scores for the control group) with a theoretical range from 1 (one point difference in coding on a single dimension) to 16 (maximum difference between groups on all four coded dimensions).

Similarly, we determined and reflected the number of dimensions with differential scores higher than zero (i.e., on how many dimensions adaptive qualities of the instruction were present in the experimental group to a greater extent than in the control group). This variable is labeled “Difference by Dimension” and could range from 1 (difference on a single dimension) to 4 (difference on all four dimensions), regardless of the magnitude of that difference.

Finally, we wanted to trace and analyze the source of the difference. To that end, a categorical variable Source of the Difference was designed to reflect what dimensions in what combinations contributed to the magnitude of the respective effect size. Initial letters of each of the coded dimension depicted levels of this variable. For example, F_T_A stands for some difference on the dimensions of Flexibility, Teacher's Role, and Adaptability of Instruction, with zero differential score on the dimension of Pacing.

Decisions about the completeness of the reported information were made at three points in time. First, two independent coders decided in general on each study's inclusion/exclusion status (overall qualitative judgment). Second, when dimensions were actually coded, reviewers searched for all relevant information in the study and if this information was not found, assigned the valued of “999” (or missing information) subsequently excluding studies with more than one “999.” Third, at the analysis stage studies with “999” were converted into zeroes, indicating no difference between the two respective conditions, and if after this transformation the overall composite score was zero these studies were also excluded.

As a result, only studies judged by coders to have provided sufficient description were retained for analysis.

4.1.4 Types of outcome measures

4.1.5 Duration of follow-up

To maximize coverage of primary research, fully compatible in terms of outcome measures, only immediate post-test results were considered. Various forms of delayed post-tests were documented and their time lags categorized to inform further reviews.

4.1.6 Types of settings

As stated earlier, k-12 formal educational settings (∼ages 5–18), in educational programs leading to advancement to the next academic level/grade, were required in the current meta-analysis. Other settings (i.e., homeschooling, auxiliary programs, summer camps, vocational workshops, etc.) were excluded.

4.2.1 Electronic searches

Following the Guidelines of the Campbell Collaboration (Kugley et al., 2017), in order to retrieve a broad base of studies to review we started by having an experienced Information Specialist search across an array of bibliographic databases, both in the subject area and in related disciplines. The following databases were searched for relevant publications: ABI/Inform Global (ProQuest), Academic Search Complete (EBSCO), ERIC (EBSCO), PsycINFO (EBSCO), CBCA Education (ProQuest), Education Source (EBSCO), Web of Knowledge, Engineering Village, Francis, ProQuest Dissertations & Theses Global, ProQuest Education Database, Linguistics and Language Behavior Abstracts (ProQuest).

The search strategy was tailored to the features of each database, making use of database-specific controlled vocabulary and search filters. Searches were limited to the year 2000–2017, and targeted a k-12 population. The following is an example from the ERIC database:

(AB (adaptive OR personalized OR personalized OR individuali*) AND AB (pedagog* OR learning OR teaching OR instruction OR education OR classroom OR curriculum)

OR (AB “self direct*” OR “self regulate*”) OR (DE “Open Education” OR DE “Discovery Learning” OR DE “Individual Activities” OR DE “Student-Centered Curriculum” OR DE “Student Centered Learning” OR DE “Mastery Learning” OR DE “Independent Reading” OR DE “Independent Study” OR DE “Individualized Instruction” OR DE “Competency-Based Education” OR DE “Individual Instruction” OR DE “Individualized Programs” OR DE “Individualized Reading” OR DE “Individualized Transition Plans” OR DE “Learner Controlled Instruction” OR DE “Pacing” OR DE “Individual Testing” OR DE “Adaptive Testing” OR DE “Experiential Learning” OR DE “Learner Engagement” OR DE “Cooperative Learning”))

AND

(DE “Program Validation” OR DE “Academic Achievement” OR DE “Instructional Improvement” OR DE “Progress Monitoring” OR DE “Educational Assessment” OR DE “Instructional Effectiveness” OR DE “Program Evaluation” OR DE “School Effectiveness” OR DE “Evidence” OR DE “Outcomes of Education” OR DE “Program Effectiveness”)

AND

(DE “Pretesting” OR DE “Pretests Posttests” OR DE “Control Groups” OR DE “Experimental Groups” OR DE “Matched Groups” OR DE “Mixed Methods Research” OR DE “Randomized Controlled Trials” OR DE “Effect Size” OR DE “Quasiexperimental Design” OR DE “Comparative Analysis”)

Limiters—Date Published: 20000101-20171231; Educational Level: Elementary Education, Grade 1, Grade 2, Grade 3, Grade 4, Grade 5, Grade 6, Grade 7, Grade 8, Grade 9, Grade 10, Grade 11, Grade 12, High Schools, Junior High Schools, Kindergarten, Middle Schools, Primary Education, Secondary Education; Publication Type: Books, Collected Works (All), Dissertations/Theses (All), ERIC Publications, Information Analyses, Journal Articles, Numerical/Quantitative Data, Reports—Descriptive, Reports—Evaluative Reports, Reports—Research.

4.2.2 Searching other resources

4.3.1 Selection of studies

The overall set of inclusion/exclusion criteria for the meta-analysis contained the following requirements:

Be publicly available (or archived) and encompass studies from 2000 to the present.

4.3.2 Data extraction and management

Two researchers independently conducted abstract screening and full-text review of studies identified through the whole complex of searching activities, compared notes, discussed and resolved disagreements, and documented reliability rates. Similar procedures were employed for effect size extraction and coding of moderator variables.

4.3.3 Effect size extraction and calculation

One of the selection criteria was “Contain sufficient statistical information for effect size extraction,” so that an effect size could be calculated for each independent comparison. This information could take several forms (in all cases sample size data were required):

Means and standard deviations for each treatment and control group;

Effect sizes were initially calculated as Cohen's d and then converted to Hedges’g (i.e., correction for small samples). Standard errors (

S E d

d ¯

The effect sizes were coded for precision of calculations and analyzed in moderator variable analysis.

4.3.4 Description of methods used in primary research

True experimental and quasi-experimental studies were included as far as they feature two educational interventions covering the same content (required knowledge acquisition and/or skill development). They were assessed on compatible outcome measures, where one group (experimental) is greater in Student-Centered qualities (as described earlier) compared to the other (control) group with fewer Student-Centered qualities. Reporting quantitative data sufficient for an effect size extraction was a necessary condition for study inclusion.

4.3.5 Criteria for determination of independent findings

There are several potential major threats to the independence of the findings. These are: (a) Repeated use of data coming from the same participants (i.e., dependence); (b) reporting multiple outcomes of the same type; and (c) aggregating outcomes of different types representing the same sample of participants (does not apply to this review, as it is limited to learning achievement outcomes only). The means that we used for ensuring data independence were that no group of participants was used more than once, resulting in most cases in only one effect size per study; and only one outcome measure was used in each comparison (either cumulative or composite achievement score).

4.3.6 Details of study coding categories

In addition to the coding dimensions of Student-Centered pedagogical qualities that would determine proper comparisons for effect size extraction, the following groups of study coding categories were used in the review. First, study methodological quality was assessed for features such as design type, the fidelity of treatment implementation, attrition, and the unit of assignment/analysis (Cooper, Hedges, & Valentine, 2009). Within the same category, we coded for outcome source and psychometric quality of the assessment tools, as well as for the precision of procedures used for effect size extraction and for equivalence of instructor and study materials. Jointly, these methodological study features were used in moderator variable analyses to inform us of any potential threats to all types of study validity (Cooper et al., 2009).

Substantive study features further clarify descriptions of Student-Centered pedagogical qualities by specifying theoretical models underlying instructional practices under review, treatment duration, instructor's experience, provision of professional development for teachers, and training for students, whenever it is required by specific instructional intervention. Demographic study features encompass learners’ age, educational background, and ability level, as well as, subject matter studied. All of these study features were subsequently analyzed as moderators for their potential impact on treatment effects.

4.3.7 Assessment of risk of bias in included studies

Assessment of the risk of bias was accomplished in several ways:

4.3.8 Sensitivity analysis

Sensitivity analysis was performed to determine if issues such as research design, effect size extraction methods, instructor and material equivalence, publication bias, and assessment tool category, might have introduced bias into the results. It also involves a “one study removed” analysis of the distribution effect sizes. For the results of this analysis please see Table 5a–e.

4.3.9 Assessment of reporting biases

Publication bias was assessed based on the examination of a Forest Plot and associated “Trim and Fill” analysis plus other tests such as Classic Fail-safe analysis and Orwin's Fail-safe N (Orwin, 1983).

4.3.10 Assessment of heterogeneity

Homogeneity assessment, sometimes called an analysis of precision, was accomplished using the fixed model of analysis. The following indicators are reported and discussed:

Q-Total, df, test of the null hypothesis, I ² (percentage of error variance over and above chance), and tau² (average variability used in the calculation of random weights).

4.3.11 Data synthesis

Data are synthesized, initially, under the random effects model, and includes the following statistics: Overall weighted random effects analysis with the statistics of

g ¯

The protocol for this review was published in the Campbell Library, August 2016: https://campbellcollaboration.org/media/k2/attachments/Bernard_Operationalized_Adaptive_Teaching_Title.pdf

5.1.1 Results of the searches

All searches were conducted by a fulltime Information Specialist (MLS level) and member of the Systematic Review Team at the Centre for the Study of Learning and Performance at Concordia University in Montreal, QC, Canada. As shown in the PISA flowchart in Figure 1, there were three sources of studies: (a) 1,663 studies from dedicated bibliographic searches detailed in the Method; (b) 95 studies retrieved from the grey literature; and (c) 254 studies transferred from an internal database of studies retrieved for a larger meta-analysis that includes all grade levels, but with the same inclusion/exclusion criteria as the current study. Figure 1 details the results at each stage of the search and retrieval process. All bibliographic information was exported into an Endnote database and managed from there.

OPEN IN VIEWER

1

Flow diagram of the review process

OPEN IN VIEWER

2

Venn diagram of the overlap among methods of active learning (Student-Centered; Bishop & Verleger, 2013, p. 6)

Duplicate studies were removed (n = 247) and the remaining 1,765 studies were subjected to an abstract screening process. In all, 817 studies were retrieved as full-text documents. Examination of these studies proceeded according to the details described in the Method. A total of 518 full-text documents were excluded for reasons detailed in the inclusion/exclusion description in the Method, leaving 299 studies that were included in the final analysis. In the final stage, 365 independent effect sizes were extracted from these studies, coded, and analyzed.

There are 365 effect sizes (299 individual studies) included in this review, representing 43,175 treatment and control participants. References to these 299 studies appear in the section entitled References to included studies. Please see Table S13 for complete statistical information for the 365 effect sizes.

5.1.3 Excluded studies

A total of 1,613 studies were excluded from this review. Figure 1 shows how this number diminished over the course of the review and selection process and references to these excluded studies are presented in the section entitled References to excluded studies (found in Online Supplement 1).

5.2.1 Publication bias analysis

Borenstein et al. (2014) state:

“The basic issue of publication bias is that not all completed studies are published, and the selection process is not random (hence the “bias”). Rather, studies that report relatively large treatment effects are more likely to be submitted and/or accepted for publication than studies [that] report more modest treatment effects. Since the treatment effect estimated from a biased collection of studies would tend to overestimate the true treatment effect, it is important to assess the likely extent of the bias, and its potential impact on the conclusions” (Publication Bias Report, Comprehensive Meta-Analysis, 2014).

Thus, this report includes an extensive investigation of publication bias, as a potential source of difficulty and error in interpreting these results.

Funnel Plot analysis and Trim and Fill

A Funnel Plot (See Figure 3) and associated Trim and Fill procedure (Duval & Tweedie, 2000) of 365 effect sizes (See Table 3) indicate that there is no discernable publication bias on the negative side of the plot (i.e., left of the mean effect size) under the random effects model.

OPEN IN VIEWER

3

Random effects funnel plot (effect size by standard error). [Color figure can be viewed at wileyonlinelibrary.com]

OPEN IN VIEWER

Table 3

Analysis of publication source

Categories	k	g ¯	SE	Lower 95th	Upper 95th	z-value	p value	Q-B	df	p value
Journal articles	302	0.46	0.03	0.40	0.53	13.69	.00
Theses	8	0.31	0.12	0.08	0.54	2.60	.01
Other	54	0.35	0.07	0.21	0.50	4.85	.00
Total between								3.12	2	.21

Another indicator, Classic fail-safe N, suggests that 121,993 additional effect sizes would be needed to bring the observed p value below alpha = .05 (i.e., 860 additional “null” effect sizes per each observed effect size). Also, Orwin's fail-safe N (Orwin, 1983), suggests that 125 additional “null” effect sizes would be needed to bring the observed average effect size to a trivial level of

g ¯

= 0.10.

In addition, an analysis of publication type (See Table 3) indicated that journal articles (

g ¯

 = 0.46, k = 302), unpublished theses (

g ¯

 = 0.31, k = 8) and other unpublished documents (e.g., conference papers), (

g ¯

 = 0.35, k = 54) were not significantly different in mixed-model moderator analysis (Q-Between = 3.12, df = 2, p = .21). However, these nonsignificant findings should be viewed cautiously because of the small k for theses, possibly resulting from an issue of power.

Overall, there appear to be no serious issues of bias related to the analysis of published data.

5.2.2 Sensitivity analysis

Sensitivity analysis examines issues in the data and coding that might affect the reliability of the results. First, we conducted a one study removed (Borenstein et al., 2014, CMA, Version 3.3.070) analysis of effect size and study sample on the variability of the individual data points across the distribution. Table 4 shows partial results of that analysis. The table contains six studies from the top of the distribution (highest effect sizes) and six studies from the bottom (lowest effect sizes, all negative). There is only a 0.01th difference in average effect size between the top and the bottom of the distribution when each study is removed sequentially. Also, the standard errors and the limits of the 95th confidence demonstrate the same consistency. Since the most problematic studies often reside on the peripheries of the distribution, large/small in effect size magnitude and large in sample size (i.e., high influence studies), the relative random weights were included in the last column. In the 12 studies displayed, their influence ranged from 0.13–0.28 on the upper end and 0.19–0.33 on the lower end indicating little concern for undue influence.

OPEN IN VIEWER

Table 4

Sample (top six studies and bottom six studies) of one study removed

Study names	Actual	One study removed				Relative wt.
	g	g ¯	SE	Lower 95th	Upper 95th
1. Zohar_2 2008	3.10	0.44	0.03	0.38	0.50	0.20
2. Lamidi 2015	3.10	0.44	0.03	0.38	0.49	0.28
3. Garcâ_1 2006	3.10	0.44	0.03	0.38	0.50	0.26
4. Ben-David 2009	3.10	0.44	0.03	0.38	0.49	0.27
5. Alfassi_1 2003	3.10	0.44	0.03	0.38	0.50	0.15
6. Alfassi_2 2003	2.85	0.44	0.03	0.38	0.50	0.13
360. Eysink_2 2009	−0.88	0.45	0.04	0.23	0.39	0.33
361. Furtak_1 2012	−0.97	0.45	0.04	0.23	0.39	0.19
362. Chang_2 2002	−0.97	0.45	0.04	0.23	0.39	0.25
363. Sola_2 2007	−0.98	0.45	0.04	0.23	0.39	0.29
364. Wesche 2002	−1.07	0.45	0.04	0.23	0.39	0.32
365. Bassett 2014	−1.48	0.45	0.04	0.23	0.39	0.25
Overall (k = 365)	0.44	0.44	0.03	0.38	0.50	100%

Several issue related to study design quality and methodology (See Table 5a–c) also suggest no or minimal potential bias in these results (data below reported according to the mixed model analyses):

Bias in research design (Table 5a)—in moderator variable analysis, quasi-experimental research designs versus randomized control trials, show no significant difference in average effect sizes (

g ¯

 = 0.46, k = 273 and

g ¯

 = 0.40, k = 90, respectively, Q-between = 0.55, df  = 1, p = .46);

Effect size extraction bias (Table 5b)—effects calculated from exact descriptive statistics (e.g., means and SDs, exact t-values, exact p values) versus effects estimated based on other statistics with the element of some assumptions (e.g., reported nonexact significance levels, p < .05) indicates no observable bias (

g ¯

 = 0.37, k = 107 vs.

g ¯

 = 0.48, k = 257, respectively, Q-between = 2.83, df = 1, p = .09).

Bias in the quality of instrument types (Table 5c)—standardized tests versus modified or piloted standardized tests versus teacher/researcher made tests versus combinations of measure types indicates no bias in average effect sizes (

g ¯

 = 0.37, k = 78, versus

g ¯

 = 0.44. k = 79, versus

g ¯

 = 0.48, k = 196, vs.

g ¯

 = 0.35 vs. k = 11, respectively, Q-between = 3.11, df = 3, p = .37).

Bias in coding—Five sources of coding bias were recorded and are presented here as percentage of agreement and Cohen's Kappa (κ; i.e., inter-rater reliability):

1.

Abstract screening, 84.48% or κ = 0.69;

2.

Full-text review, 96.08% or κ = 0.92;

3.

Decisions on number of effects per study, 92.78% or κ = 0.92;

4.

Data extraction and effect size calculation, 96.21 or κ = 0.92; and

5.

Study Feature coding (including the four primary dimensions), 92.23% or κ = 0.84.

OPEN IN VIEWER

Table 5

a–e Research design, extraction method, instrument type, and publication source

Codes	k	g ¯	SE	Lower 95th	Upper 95th	z-value	p value	Q-B	df	p value
Research design—a
QED	273	0.46	0.03	0.39	0.53	13.18	.00
RCT	90	0.40	0.07	0.28	0.53	6.17	.00
Total between								0.55	1	.46
Extraction method—b
Exact	107	0.37	0.05	0.27	0.47	7.22	.00
Approximate	257	0.48	0.04	0.40	0.55	12.69	.00
Total between								2.83	1	.09
Instrument type—c
Standardized	78	0.37	0.00	0.28	0.46	7.95	.00
Mod. Stand.	79	0.44	0.01	0.30	0.59	5.95	.00
Teacher/Rcher.	196	0.48	0.00	0.39	0.57	10.26	.00
Combo	11	0.35	0.02	0.10	0.60	2.78	.01
Total between								3.11	3	.37
Teacher class assignment—d
Same teacher	156	0.32	0.00	0.23	0.42	6.64	.00
Diff. teacher	196	0.52	0.00	0.44	0.60	13.12	.00
Total between								10.19	1	.00
Teacher training—e
No	128	0.50	0.05	0.40	0.60	9.86	.00
Yes	208	0.45	0.04	0.37	0.53	10.86	.01
Total between								0.63	1	.43

Note: Missing data has been removed so k does not always equal 365.

Abbreviations: QED: quasi-experimental design; RCT: randomized control trial.

These coding values are deemed to be within normal range and so no bias seems to be present.

Two other potential sources of bias that arose from classroom conditions were also tested:

Bias in teacher assignment (Table 5d)—same teacher in both classrooms versus different teachers in each classroom (

g ¯

 = 0.32, k = 157 vs.

g ¯

 = 0.52, k = 196, Q-between = 10.19, df  = 1, p = <.001. In this case, bias seems to be present, with different teachers in each classroom outperforming classrooms where the same teacher was assigned.

Teacher training in Student-Centered methods (Table 5e)—no teacher training versus teacher training (

g ¯

 = 0.50, k = 218 vs.

g ¯

 = 0.45, k = 208, Q-between = 0.63, df = 1, p = .43). There appears to be no bias due to differential teacher training.

Overall, this assessment of methodological and classroom variables indicates only one area of concern: The same teacher assigned to both the treatment and control conditions or a different teacher assigned to each condition. Different teachers appear to produce significantly higher effect sizes than when the same teacher is used in the two conditions. While this form of bias is of concern by itself and was further explored in the subsequent analyses, in light of all of the other bias issues tested and found to be equivalent in their influence on the treatment effect, it is unlikely that this issue alone affected the ultimate conclusions of this review.

5.3 Synthesis of results

5.3.1 Primary analysis

Basic question

The first question involved the overall average effect on achievement outcomes of more adaptive instruction as it is reflected in the difference between more Student-Centered instructional conditions (the treatment condition) and less Student-Centered conditions (the control condition). It is important to understand that this is not necessarily a contrast between Student-Centered classrooms and Teacher-Centered classrooms. It is instead the differential in ratings (on four effect size-defining dimensions outlined in the Method section) between a treatment (more Student-Centered) and control condition (less Student-Centered) that range from equal (i.e., zero, treatment and control are equally Student-Centered and Teacher-Centered) to large, (i.e., up to +3 or +4—a theoretical, not necessarily observed in this review range, Student-Centered is much greater than Teacher-Centered).

In all, 365 effect sizes are included in the meta-analysis. Four very large effect sizes (>4.00) are adjusted (Winsorized; Hastings, Mosteller, Tukey, & Winsor, 1947) to match the next lower effect size, the fifth largest, in the distribution (g = 3.1). This produces a change in the mean effect size of 0.10 and a similarly slight adjustment to the other statistics. There are no outliers at the negative end of the distribution.

The results of this analysis (See Table 6 for both the unadjusted and adjusted statistics) produces a significant weighted adjusted average effect size of

g ¯

 = 0.444, k = 365, SE = 0.03, z = 14.56, p < .000. The distribution is significantly heterogeneous (Q-Total = 3095.89, df = 364, p < .0001, with an I ² value of 88.22 and a τ ² (tau-squared) of 0.27. This result suggests that on average more Student-Centered classroom studies produce better results on achievement outcomes than do less Student-Centered classroom studies. The average weighted effect size is of moderate size (Cohen, 1988) and indicates that on average the more C-S condition (treatment) outperformed the less C-S (control) by 0.444sd. The average effect size is used as a reference point to describe the collection of student-centered versus teacher-centered practices when compared and reflects the overall benefit for learning when student-centered qualities are present. The subsequent moderator variable analysis (presented in the next section) attempts to explain the extent to which each dimension contributes (or does not contribute) to the overall average.

OPEN IN VIEWER

Table 6

Overall results for unadjusted and Winsorized data sets

Model	Effect size and 95th confidence interval					Test of null
Random effects	k	g ¯	SE	Lower 95th	Upper 95th	z-value	p value
Unadjusted	365	0.454	0.03	0.39	0.52	14.44	<.00
Winsorized	365	0.444	0.03	0.38	0.50	14.56	<.00

Model	Heterogeneity
Fixed effect	Q-value	df	p value	I 2	Tau 2
Unadjusted	3,327.78	364	<.00	89.03	0.29
Winsorized	3,095.89	364	<.00	88.22	0.27

Simple meta-regression can provide a sense of this relationship between the strength of Student-Centered and achievement outcomes. A moderator variable reflecting the degree of Student-Centered (the quantitative differences between the ratings of the treatment/control) was created to test this relationship. If this relationship is patterned (either positively or negatively) rather than irregular, the result of the meta-regression of achievement on the degree of student centeredness should result in a positive, significant slope. If the slope of the regression line is not positive and significant, indicating the absence of a positive linear progression, we can assume that the relationship between student centeredness and achievement results is irregular, thereby diminishing the argument that more Student-Centered classrooms are more advantageous to the attainment of achievement outcomes than less Student-Centered Classrooms.

The simple meta-regression of the relative difference between more Student-Centered and less Student-Centered resulted (the defining characteristic of the treatment-control contrast) in a significant slope (β = 0.037, SE = 0.017, z = 2.14, p = .03). The test of the model resulted in Q-Between = 4.58, df = 1, z = .03. Q-within is also significant. These results indicate a marginally positive relationship between the degree of student centeredness and the achievement of learning outcomes by students. At best, this result is considered to be a weak but positive effect. Complete results of this analysis can be found in Table 7.

OPEN IN VIEWER

Table 7

Overall strength of the relationship between treatment and control (degree of student-centeredness)

Covariate	β	SE	Lower 95th	Upper 95	z-value	p value	VIF
Intercept	0.34	0.06	0.22	0.45	5.76	<.001	3.67
Degree of Student-Centered instruction	0.04	0.02	0.00	0.07	2.41	.032	1.00
Test of model: Q = 4.58, df = 1, p = .032. Goodness of Fit: Q = 3094.41, df = 363, p < .000, tau 2 = 0.27

Note: This and all subsequent meta-regression analyses use Random Effects Method-of-Moments Model.

5.3.2 Primary predictor variables

There are four primary predictor variables that represent the degree of the Teacher's Role, Pacing, Flexibility, and Adaptivity that is offered to students. Studies are coded as a differential between the treatment and control conditions in terms of less flexible/adaptive classroom practices or more flexible/adaptive practices. These differentials form a hypothetical continuous integer-level scale ranging from −4 to −1 for more Teacher-Centered practices (less flexible/adaptive) and +1 to +4 for more Student-Centered practices (more flexible/adaptive), with 0 (zero) interpreted as equality between the control and treatment conditions. Please, keep in mind that though negative-to-positive fluctuations within each dimension are theoretically possible, only studies whose total differential score (the sum of four dimensions) is positive (i.e., overall in favor of Student-Centered qualities of instruction) were retained in our meta-analysis. There are four dimensions of classroom practice (these are described in detail in the Method) that were identified and to which this coding was applied:

Teacher's role as a lecturer/guide/mentor;

Pacing of instruction to meet student needs/preferences;

Flexibility in the creation/use of study materials, course design, etc.;

Adaptability of feedback and learning activities to students, individual interests of students, etc.

The question being asked in this moderator variable analysis is which, if any, of these classroom practices, predicts levels of effect size. Initially, meta-regression is used to explore this question. Then, treating the scale as categorical data, the various levels are explored through mixed moderator variable analysis (i.e., ANOVA-analog). Finally, combinations of these dimensions are explored to determine if they can better characterize the totality of the instruction.

Meta-regression of dimensions (primary moderator variables)

Initially, all four dimensions were entered into multiple meta-regression (random effects method of moments) in the order that they are described above. The dependent or outcome variable in this analysis was the effect sizes of individual studies (k = 365).

The overall model, excluding the intercept (See Table 8a), was significant (Q-Between = 31.02, df = 4, p < .001). The extent of unexplained variation goodness of fit test for heterogeneity was also significant (Q-within = 2,912.94, df = 360, p < .001, I ² = 87.64%, Tau ² = 0.2615, Tau = 0.5113). Two of the four moderator variables were significant predictors of effect size (though acting in opposite directions): Pacing (β = −0.1542, SE = 0.045, z = −3.45, p = .0006); and Teacher's role (β = 0.154, SE = 0.039, z = 3.96, p = .0001). The other two predictor variables, Flexibility and Adaptability, were not significant.

OPEN IN VIEWER

Table 8

a–b Meta-regression results for interval-level moderator variables

Covariates		β		SE		Lower 95th		Upper 95		z-value		p value		VIF
Meta-regression of four predictors—a
Intercept		0.33		0.06		0.21		0.45		5.38		<.001		4.11
Pacing		−0.15		0.04		−0.24		−0.07		−3.45		<.001		1.04
Teacher's role		0.15		0.04		0.08		0.14		3.96		<.001		1.06
Adaptability		0.06		0.04		−0.02		0.14		1.52		.13		1.04
Flexibility		0.04		0.04		−0.04		0.11		0.99		.32		1.07
Test of model: Q = 31.02, df = 4, p < .001. Goodness of Fit: Q = 2921.91, df = 360, p < .0001, tau 2 = 0.51
Meta-regression of two predictors (reduced model)—b
Intercept		0.36		0.06		0.25		0.47		6.21		<.001		3.74
Pacing		−0.14		0.04		−0.23		−0.05		−3.18		.002		1.01
Teacher's role		0.17		0.04		0.09		0.24		4.42		<.001		1.00
Test of model: Q = 27.74, df = 2, p < .001. Goodness of Fit: Q = 2920.95, df = 362, p < .0001, tau 2 = 0.51

Abbreviation: VIF: variance inflation factor.

The analysis was re-run (See Table 8b) with the two nonsignificant variables removed. This model was also significant (Q = 27.74, df = 2, z = 4.42 p < .001) and heterogeneous (Q-within = 2,920.95, df = 362, p < .001, I ² = 87.61%, Tau ² = 0.2583, Tau = 0.508). The variables in the reduced model were both significant: Pacing (β = −0.139, SE = 0.044, z = −3.16, p = .0015); and Teacher's role (β = 0.167, SE = 0.038, z = 4.42, p < .0001). It is interesting that the moderator Pacing is a negative predictor of effect size, while Teacher's role is positive. Note that in all of these analyses the variance inflation factor is low, indicating a lack of collinearity (Thompson & Higgins, 2002). See Figures 4, 5 for scatterplots of these results.

OPEN IN VIEWER

4

Scatterplot of Teacher's Role from meta-regression (Table 7b). [Color figure can be viewed at wileyonlinelibrary.com]

OPEN IN VIEWER

5

Scatterplot of Pacing from meta-regression (Table 7b). [Color figure can be viewed at wileyonlinelibrary.com]

To examine these results from another perspective, mixed moderator variable analysis was conducted for Pacing, Teacher's Role, Adaptability, and Flexibility, each explored across levels of differential scores (Table 9a–d). Several categories with cell frequencies (number of cases per level of the category) less than 5 were removed from each of these analyses. The same variables, Pacing, Teacher's Role, are significant across the categories of relative scores and the same pattern of —Teacher's Role, as a positive predictor of effect size, and Pacing as a negative predictor of effect size.

OPEN IN VIEWER

Table 9

a–d Comparison of levels of relative strength (how much Student-Centered) for four primary moderator variables

Levels	k		g ¯	SE	Lower 95th	Upper 95th	z-value	p value	Q-Bet.	df	p value
Teacher's Role—a
0 (Student-Centered = Teacher-Centered)	80		0.31	0.07	0.17	0.44	4.51	.00
1 (Student-Centered > Teacher-Centered)	189		0.41	0.04	0.33	0.49	9.68	.00
2 (Student-Centered >> Teacher-Centered)	79		0.58	0.06	0.46	0.69	10.07	.00
3 (Student-Centered >>> Teacher-Centered)	15		0.78	0.21	0.37	1.20	3.72	.00
Total between									12.65	3	.01
Pacing—b
0 (Student-Centered = Teacher-Centered)	151		0.54	0.05	0.44	0.64	10.59	.00
1 (Student-Centered > Teacher-Centered)	172		0.40	0.04	0.32	0.48	9.58	.00
2 (Student-Centered >> Teacher-Centered)	38		0.22	0.08	0.06	0.38	2.72	.01
Total between									12.37	2	.002
Adaptability—c
0 (Student-Centered = Teacher-Centered)	179		0.38	0.04	0.31	0.46	9.94	.00
1 (Student-Centered > Teacher-Centered)	140		0.49	0.06	0.38	0.60	8.57	.00
2 (Student-Centered >> Teacher-Centered)	39		0.55	0.10	0.36	0.73	5.66	.00
3 (Student-Centered >>> Teacher-Centered)	6		0.41	0.15	0.12	0.70	2.77	.01
Total between									3.99	3	.26
Flexibility—d
0 (Student-Centered = Teacher-Centered)	261		0.42	0.03	0.35	0.48	12.23	.00
1 (Student-Centered > Teacher-Centered)	64		0.47	0.09	0.29	0.65	5.18	.00
2 (Student-Centered >> Teacher-Centered)	29		0.60	0.11	0.40	0.81	5.76	.00
3 (Student-Centered >>> Teacher-Centered)	10		0.61	0.25	0.11	1.10	2.41	.02
Total between									3.48	3	.32

There is nothing unique in this later analysis, but it does give a clear sense of the magnitude of effect within each category of relative strength. For example, Teacher's Role ranges in effect size magnitude from

g ¯

 = 0.78 at strength level 3 to

g ¯

 =  0.31 at level 0 (Table 9a), where the treatment and control conditions are equally balanced. Pacing (Table 9b) moved in the opposite with and average effect at strength 2 of

g ¯

 = 0.22 and at

g ¯

 = 0.54 at strength level 0. Both of these variables are significant in this analysis of categories.

By contrast, neither of the other two variables (Table 9c,d), Adaptability and Flexibility, is patterned or significant across levels of relative strength. For the two variables, average effect sizes ranges from

g ¯

 = 0.38 to

g ¯

 = 0.61, a relatively short range compared to Teacher's Role and Pacing.

The next question we asked concerns the combinations of these four variables. In this analysis, dimensions alone are compared with the dimensions paired (i.e., Teacher's Role + Pacing). The two significant predictors, Teacher's Role and Pacing are shown in Table 10. The between-group z-value comparing the two alone and their pair is not significant (p = .44).

OPEN IN VIEWER

Table 10

Teacher's role and pacing and their combination

Levels	k	g ¯	SE	Lower 95th	Upper 95th	z-value	p-value	Q-Bet.	df	p value
Teacher's role, pacing and combination
Teacher's role	61	0.48	0.07	0.34	0.62	6.83	.00
Pacing	5	0.50	0.15	0.20	0.80	3.26	.00
Teacher + Pacing	14	0.72	0.25	0.23	1.20	2.91	.00
Total between								2.69	3	.44

However, when the other two variables, Flexibility, and Adaptability, are examined in the same way (single dimensions and pairs), the between-group z-value is significant (p = .01; See Table 11a). The two extremes with reasonable cell frequencies are further tested in post hoc analysis in Table 11b. These are Teacher's Role paired with Flexibility (k = 44,

g ¯

 = 0.31) and Teacher's Role paired with Adaptability (k = 33,

g ¯

 = 0.66). The overall Q-between is significant (z = 7.58, p = .006).

OPEN IN VIEWER

Table 11

a–b Two variables (adaptability and flexibility) and their combinations

Levels	k	g ¯	SE	Lower 95th	Upper 95th	z-value	p value	Q-Bet.	df	p value
Adaptability, Flexibility and combinations—a
Adaptability	19	0.23	0.11	0.02	0.44	2.10	.04
Flexibility	29	0.19	0.09	0.02	0.35	2.17	.03
Flexibility + Adaptability	15	0.35	0.24	−0.12	0.82	1.47	.14
Flexibility + Teacher's role	44	0.31	0.08	0.15	0.46	3.89	.00
Adaptability/Pacing	6	0.34	0.28	−0.21	0.89	1.20	.23
Adapt. +Teacher's role	33	0.66	0.10	0.46	0.86	6.57	.00
Total between								14.55	5	.01
Teacher's role and combinations with Flexibility and Adaptability (post hoc)—b
Teacher's role	61	0.48	0.07	0.34	0.62	6.83	.00
Flexibility + Teacher's role	44	0.31	0.08	0.15	0.46	3.89	.00
Adapt. + Teacher's role	33	0.66	0.10	0.46	0.86	6.57	.00
Total between								7.76	2	.02

Note: The total number of all single dimensions and combinations is k = 365. Some have been excluded (k = 226).

Demographic moderator variable analysis

Moderator variables in this study (Table 12a–d), beyond those already described, are mostly demographic in nature. Thus, they are less important to the main focus but they do give a sense of the range of conditions that exist within the data set. In Table 12a–d, it is interesting that only one demographic variable is significant—Ability Profile and that only one is close to significance (STEM vs. Non-STEM; Table 12b). The first is a contrast of two categories with reasonable cell frequencies greater than five, General Population and Special Education. The effect sizes were

g ¯

 = 0.42 (k = 338) and

g ¯

 = 0.80 (k = 26) in favor of Special Education. Non-STEM subjects outperformed STEM in absolute magnitude, but the contrast was not significant (

g ¯

 = 0.52 vs. 0.40).

OPEN IN VIEWER

Table 12

a–d Mixed moderator variable analysis for categorical demographics

Levels	k	g ¯	SE	Lower 95th	Upper 95th	z-value	p value	Q-Bet.	df	p value
Grade Level—a
Kindergarten	7	0.44	0.18	0.10	0.79	2.52	.01
Gr. 1–5	116	0.36	0.04	0.28	0.45	8.38	.00
Gr. 6–8	95	0.47	0.07	0.34	0.60	6.93	.00
Gr. 9–12	124	0.50	0.06	0.38	0.62	8.15	.00
Total between								4.11	3	.39
Subject matter (Non-STEM vs. STEM)—b
Non-STEM	93	0.52	0.07	0.39	0.65	7.93	.00
STEM	260	0.40	0.03	0.33	0.46	11.36	.00
Total between								2.80	1	.09
Subject matter (Detailed)—c
ICT	13	0.36	0.17	0.03	0.69	2.17	.03
Language Arts	52	0.48	0.06	0.35	0.60	7.57	.00
Math	80	0.37	0.06	0.26	0.48	6.61	.00
Science	168	0.43	0.05	0.33	0.52	8.94	.00
Second Lang.	12	0.57	0.18	0.21	0.92	3.11	.00
Social Sciences	12	0.64	0.27	0.03	0.69	2.33	.02
Total between								2.97	5	.70
Ability Profile—d
General Pop.	338	0.42	0.03	0.36	0.48	13.45	.00
Special Ed.	26	0.80	0.15	0.50	1.10	5.24	.00
Total between								5.96	1	0.01

To reiterate, complete descriptive statistics are contained in Table S13.

6 DISCUSSION

The purpose of this review is to examine the effectiveness of Student-Centered instructional practices in k-12 classes as it increases or depresses student achievement. Additionally, the study examines four dimensions of instructional practice, namely, “Teacher's role,” “Pacing of instruction,” “Flexibility of instructional activity,” and “Adaptability of instruction” for their individual and/or collective influence. In addition, five demographic moderator variables are also examined for their potential relationship to the effectiveness of Student-Centered instruction.

6.1 Summary of main results

6.1.1 Overall tests of Student-Centered instruction

Two tests are used to judge the overall effectiveness of Student-Centered instructional practices in promoting achievement outcomes in k-12 learners.

The first is a test of the overall outcome of 365 effect sizes. The results are significant, producing an average random effect of

g ¯

 = 0.444. Interpreted in terms of the normal distribution, this amounts to an increase for the more Student-Centered condition of 17.5% (67.5 – 50 = 17.5) over the less Student-Centered control condition. This result would be considered a moderate effect according to Cohen's (1988) interpretative criteria.

The second is the result of simple meta-regression of effect size on the total number of 0 to 4 codes for four dimensions for each study (e.g., Pacing- Flexibility- Teacher's role- Adaptability or 1-2-2-1 = 6, the sum of differential scores across dimensions). Each study was represented by a number with a theoretical range of 0–16. The analysis resulted in a positive and significant relationship (p = .03) suggesting that as Student-Centered totals increase, so does effect size. Taken together, these results reveal a tendency towards an advantage for more Student-Centered practices compared with less Student-Centered practices.

6.1.2 Primary moderator variables

The next question relates to the four dimensions, represented by the codes for each dimension above disassembled from the total number referred. Each code ranges from 0–4 In the example above, this would give Pacing a 1, Flexibility a 2, Teacher's role a 2, and Adaptability a 1.

The four dimensions are tested as predictors of effect size using multiple meta-regression. Two dimensions are significant, Teacher's role, and Pacing; Adaptability and Flexibility were not.

A second multiple meta-regression, including only Teacher's role and Pacing also produces a significant overall result. However, the relationship between the two predictors is opposite: Teacher's role is significant and positive, whereas Pacing is significant and negative.

The combination of these dimensions, tested using mixed moderator variable analysis reveals that Teacher's role and Adaptability is a better combination for promoting better Student-Centered achievement than Teacher's role and Flexibility (i.e., compared in post hoc analysis). This combination also exceeds the overall average effect size for more Student-Centered versus less Student-Centered instruction (

g ¯

 = 0.66 vs.

g ¯

 = 0.44).

6.1.3 Demographic moderator variables

Four demographic moderator variables were coded and the results of their analyses are described below.

Three of the moderator variables are not significant: Grade level, Subject matter (i.e., Non-STEM vs. STEM courses), and detailed Subject matter comparisons. None of these are significant across levels.

The variable Ability profile is significant in between-group analysis, with students in Special Education programs outperforming students deemed in the General Population (i.e.,

g ¯

 = 0.80 vs.

g ¯

 = 0.42). This result seems not to be surprising, given that Special Education teachers are trained to provide individual attention to students in small classroom settings.

6.2 Overall completeness and quality of the evidence

Clearly, this database does not include every single classroom study since 2000 that tested two groups. To find, much less to process literature that is potentially as large as this would be is a monumental task. Therefore, we had to be selective and limit the database in two important ways. First, we selected only studies that contained two compared groups that included enough information in each group to assess the qualities of Student-Centered that we were looking for. Second, we selected only high-quality quasi-experimental designs (QEDs) and randomized control trials (RCTs), thus further limiting the potential pool of studies. As a result, we consider this corpus of 299 studies and 365 independent effect sizes to be a reasonable representation of the larger body of studies that we either excluded or that could not be accessed.

6.3 Limitations and potential biases in the review process

6.3.1 High-inference versus low-inference coding procedures

One of the obvious limitations and a potential source of bias in this study is the fact that it uses an extensive amount of high-inference coding (Cooper, 2017). There is no treatment or control, per se, but instead, a set of judgments by reviewers, first as to the very definition of the treatment and control conditions (i.e., the treatment is the condition that is more Student-Centered and, conversely, the control is the condition judged to be less Student-Centered). These decisions by two independent coders were judged to be high in inter-rater reliability for the direction of the effect (e.g., + vs. −; κ = >0.86), and for the precision of calculation (κ = >0.92). Second, judgments were made by coders as to the exact ratings (e.g., +3 vs. +4) applied to each of the four dimensions. Again, these decisions were made by at least two coders working independently and producing inter-rater reliability of κ = 0.67. It is important to note in considering the accuracy of coding that raters/coders received extensive training for this task, including multiple practices runs on studies previously judged to have been accurately and reliably coded.

Also, it is worth noting that our research team has considerable experience with this approach to establishing the treatment and control through high inference coding, and have presented a paper on the subject at the Campbell Collaboration's Ninth Colloquium (Borokhovski, Bernard, Tamim, & Abrami, 2009) as well as included high inference coding in previously published meta-analyses. In the earliest meta-analysis (Bernard et al., 2009), we compared interaction treatments (i.e., practices that link students to each other, teachers. and content) in distance education to noninteraction treatments, and then classified them as student-student, student-teacher, student-content interactions. The inter-rater agreement for this exercise was κ = 0.71.

In a later meta-analysis (Schmid et al., 2014) of the effects of technology treatments in postsecondary education, studies were rated for the degree of technology integration. Higher integration (i.e., longer, more extensive richer in functionality use of educational technology) was deemed the treatment and lower integration was the control. In this study, the inter-rater reliability for this rating step was even higher (κ = 0.80), and in the same range as in the current study.

We recognize that this form of high-inference coding contains greater risk of bias than the standard designation of treatment/control, which is normally referred to as low-inference coding. However, we see no other way to advance research synthesis in literatures such as this one beyond relatively simple comparisons between “either this or that” comparisons like the treatment/control designations that populate the educational research literature. The alternative, of course, is for primary researchers to refine their questions, but that will take some time in coming.

6.4 Agreements and disagreements with other studies or reviews

This study is in strong agreement with much of the primary and secondary literature surrounding the question of the veracity of Student-Centered educational practices (See in particular Table 1 for a summary of Student-Centered related practices). Of the several meta-analyses that have investigated the efficacy of active learning (i.e., operationalized here as more Student-Centered learning) most have found a positive effect for it. In particular, reviews by Prince (2004), Linton, Farmer, and Peterson (2014), Burch et al. (2014), and Freeman et al. (2014) support the use of various Student-Centered strategies in different levels of educational practice. However, ours is the only meta-analysis that has approached the question in this fashion. It is also the only meta-analysis that has examined where, in the range of instructional practices, this advantage for Student-Centered instruction resides. Some of these reviews concern particular areas in postsecondary education (e.g., STEM subjects) and some are more general. The current review looks at STEM learning and individual studies beyond STEM. While not significantly different, these comparisons point to a generally positive effect across all subject areas covered in the corpus of the reviewed literature.

There are also reviews of direct instruction that have found that there are advantages for lecture-based or Teacher-Centered instruction (e.g., Stockard, et al. 2018) but it is arguable that there is a place for both forms of instruction and that it is an open question as to what the joint contributions of Teacher-Centered and Student-Centered are.

7 AUTHORS’ CONCLUSIONS

This meta-analysis provides strong evidence that Student-Centered instruction leads to improvements in learning with k-12 students. Not only is the overall random effects average medium in magnitude (

g ¯

 = 0.44), but there is also a demonstrated (subtle but significant) linear relationship between more Student-Centered classroom instruction and effect size (p = .03). Taken together, these results support the efficacy of allowing students to engage in active learning or other forms of Student-Centered as part of a comprehensive educational experience. It does not, however, diminish the potential advantages imbued by direct instruction (i.e., Teacher-Centered practices). Delivering important content and other kinds of directive information to students will always be part of ordered classroom processes. As Gersten et al. (2008) have argued, there is little evidence that classrooms are organized as purely Teacher-Centered or Student-Centered.

In regards to the principal moderator variables—Teacher's role, Pacing, Flexibility, and Adaptability, it is not surprising that Teacher's role occupies a central place in facilitating Student-Centered classrooms and that the relationship of this variable to effect size produces a significant positive linear trend. It is less understandable why Pacing produces an effect in meta-regression that is significantly negative. Apparently, the pacing of instructional events in a classroom is more productive when it is less Student-Centered than when it is more Student-Centered. It is possible that pacing is best left under the control of the teacher or at least mostly influenced by the teacher.

Flexibility and Adaptability, as tested in meta-regression, failed to produce a linear relationship with average effect size. However, it is arguable that these variables are not primary, but may play a role in combination with Teacher's Role that either enhances or diminishes achievement outcomes. Teacher's Role plus Adaptability appears to boost average effect sizes, while Teacher's Role plus Flexibility appears to diminish the average effect size. This inverted relationship is not too hard to understand if one considers the definitions of the two dimensions as they were operationalized in this study:

Flexibility is the individualized creation/use of study materials, course design; and

Adaptability is the provision of feedback to students and learning activities that are geared to the individual interests of students.

Flexibility concerns the creation/choice of learning materials, a role that students do not often assume, and Adaptability is operationalized as consideration for individual students in terms of appropriately designed learning activities and individualized feedback on those activities. Pacing, found to be a negative predictor of achievement, does not appear to interact with Teacher's Role.

7.1 Implications for practice and policy

This study does not provide specific instructions for the design and development of more Student-Centered Classrooms. However, besides the overall finding that more verses less Student-Centeredenteredness improves achievement outcomes, it does suggest where these practices might be applied most beneficially in specific domains of practice. We understand from these results that the teacher's role in creating an Student-Centered classroom is critical. Given more freedom to develop intellectually as an individual and with peers (i.e., any of a variety of group-based approaches) does appear to lead to better achievement outcomes compared to more direct forms of Teacher-Centered instruction. This is one lesson that is worth learning and enacting across the k-12 spectrum since there was no differentiation among grade levels. Similarly, there was no distinction between STEM and Non-STEM courses, nor were any of the individual subject matters reliably different from one another. This suggests a universal phenomenon that is even more pronounced in Special Education courses compared to the general population of students.

7.2 Implications for research

As we have in the past (Bernard et al., 2009; Schmid et al., 2014) we argue that new research efforts in classroom-based research move toward to more nuanced questions concerning practice. So much of classroom instruction asks the question, “Does alternative treatment X outperform classroom instruction or traditional educational practices.” There were a time and place for this either/or form of research, but as the efficacy of instructional approaches is validated, new questions about varieties of the new treatments need to be asked and answered. This quote from Bernard, Borokhovski, and Tamim (2019) expresses this sentiment in clear terms: “To use David Cook's (2009) analogy, can you imagine how far automobiles would have developed if they had always been compared to their reasonable alternative at the turn of the 20th century, the horse?” In spite of Henry Ford's declaration that “if I'd listened to my consumers, I'd have given them a faster horse” (Sherrington, 2003, p. 8), the driving public got something much better than a faster horse, largely because it was abandoned as the comparison condition (p. 20). Educational researchers should do the same.

INFORMATION ABOUT THIS REVIEW

Roles and responsibilities

The review team on this meta-analysis possesses the breadth and depth of experience suggested by the Campbell Collaboration. The recommended optimal review team composition includes (a) at least one person on the review team who has content expertise (Bernard has 7 years of elementary education experience, three in a Student-Centered school; Schmid is an Educational Psychologist focusing on instructional methods, and Waddington is an Educational Philosopher focusing on constructivist and Student-Centered theory), (b) at least one person who has methodological expertise (Bernard and Borokhovski have authored and co-authored many published meta-analyses, including five published in Review of Educational Research), and conduct workshops in M-A. methodology), and (c) at least one person who has statistical expertise (Bernard and Borokhovski both have extensive statistical experience and Bernard has taught statistical methods to M.A. and Ph.D. students for over 20 years). It is also (d) recommended having one person with information retrieval expertise (Pickup possesses an MLIS degree from McGill University, has been involved in retrieval and data management for our systematic review team for 5 years, and works as a methods reviewer for the Campbell Collaboration).

Responsibilities:

●

Content: Robert M. Bernard, Eugene Borokhovski, Richard F. Schmid, and David I. Waddington

●

Systematic review methods: Eugene Borokhovski and Robert M. Bernard

●

Statistical analysis: Robert M. Bernard and Eugene Borokhovski

●

Information retrieval: David Pickup

●

Update and revision: David Pickup and Eugene Borokhovski

SOURCES OF SUPPORT

Bernard, R. M. [PI], Borokhovski, E., Schmid, R. M., Waddington, D. I., & Pickup, D. (2016–2018). A Meta-Analysis of 21st Century Adaptive Teaching and Individualized Learning Operationalized as Specific Blends of Student-Centered Instructional Events. Jacobs Foundation and the Campbell Collaboration. Support: $50,000USD.

PLANS FOR UPDATING THE REVIEW

This review will be updated on an annual basis.

DECLARATIONS OF INTEREST

None of the authors are in conflict of interest with the goals or the outcomes of this meta-analysis.

Publication bias analysis (See Figures 3–5 and Tables 1–4)

Methodological comparisons (sensitivity analysis; See Tables 5a–e)

Overall results (See Tables 6 and 7)

Results: Substantive moderator variables (See Tables 8a–b, 9a–d, 10, and 11a–b)

Demographic moderator variables (See Tables 12a–d)

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.01 MB