The Fair Process Effect in the Classroom: Reducing the Influence of Grades on Student Evaluations of Teachers

Abstract

Research findings tend to confirm anecdotal observations that instructors’ teaching evaluations are influenced by students’ grades, making some instructors feel pressured to reduce the academic rigor of their course in an attempt to get higher evaluations. To reduce this pressure, the current study tested whether distributive justice may explain the relationship between grades and student evaluation of teaching (SET) and how the fair process effect may moderate the relationship between distributive justice perceptions regarding grades and SET. Relying on the extant literature of procedural justice, we hypothesized that when students perceive no fair process that determines their grades, then: (a) the relationship between distributive justice perceptions and SET will be stronger and (b) the indirect effect of grades on SET via distributive justice perceptions will be stronger. Conversely, under conditions of strong fair process perceptions, these relationships will be attenuated. Using a survey of undergraduates’ perceptions of course fairness, we found support for our proposed hypotheses. We discuss the implications of our findings for higher education faculty.

Keywords

student evaluation of teaching procedural justice distributive justice fair process effect grades

A concern expressed by many college faculty members with regard to the end-of-the-semester student evaluation of teaching (SET) is that these assessments are heavily influenced by the grades the students receive (Lersch & Greek, 2001; Marsh & Roche, 2000). That is, some instructors fear that students who receive lower grades might “get even” with instructors by rating them lower on the SET (Spooren, Brockx, & Mortelmans, 2013). Because of this concern and consequences resulting from receiving poor SET scores, some faculty may grade more leniently to increase positive SET as a means of improving chances for promotion and retention opportunities (Greenwald & Gillmore, 1997; Marsh & Roche, 2000; Simpson & Siguaw, 2000).

The purpose of this article is not to question the validity of SET, because (a) many studies have already addressed this issue (for recent reviews, see Clayson, 2009; Spooren et al., 2013; Uttl, White, & Gonzalez, 2016), and (b) regardless of their validity, SET continue to be used widely in academia. Instead, the goal of the current study is to examine the role that student perceptions of fairness may play in the relationship between expected course grades and SET. Specifically, we hope to apply the extant theories of organizational justice to a higher education context, with two goals: (a) to provide empirically supported evidence of this relationship and (b) to allow instructors to feel more confident knowing that when they assign the grades their students deserve, the instructors are less likely to be penalized as long as they are perceived as fair.

To achieve our objectives, we first review the previous empirical research on perceptions of distributive justice in the classroom, awarding special attention to the relationship between expected course grades, distributive justice perception, and SET, as the mediation of this path remains unclear. Distributive justice refers to whether people receive the outcomes (e.g., pay, promotion, grades) they believe they deserve, based on favorability or some standard of comparison to others (Adams, 1965; Deutsch, 1973). Next, we conduct a study that applies justice theory to the classroom context, and we specifically test whether the fair process effect (Folger, Rosenfield, Grove, & Corkran, 1979) works in the classroom context. The fair process effect suggests that fair processes (e.g., giving voice, following rules/rubrics consistently, suppressing biases) not only increase favorable reactions to decisions (e.g., satisfaction, acceptance) but also attenuate the negative effect of unfavorable and unfair outcomes on reactions to decisions (Van den Bos, 2005). Therefore, we test whether the presence of perceived fair course processes attenuates the well-known association between expected course grades and SET. This question is worthy of empirical investigation, because if the fair process effect does extend to the classroom, then the correlation between expected course grades and SET (Crumbley, Flinn, & Reichelt, 2010; Simpson & Siguaw, 2000) can be attenuated through course design and implementation. Another practical benefit of empirically observing this relationship would be allowing instructors to be more at ease when employing rigorous, fair practices, and assigning grades their students deserve (e.g., low-performing students receive low grades), without fearing that their SET scores will be significantly compromised.

Literature Review

We begin our review by summarizing the evidence that expected course grades are associated positively with SET. Next, we discuss evidence that suggests a major reason this association exists is because students think of grades in justice terms, and thus distributive justice likely mediates this relationship (i.e., Grades → DJ → SET). Once this Grades → DJ → SET path is theoretically established, we turn to how instructors might attenuate this relationship; specifically, we apply the fair process effect from the management literature as a possible attenuator of the Grades → DJ → SET relationship. We explain why the fair process effect should operate in the classroom context. Based on these literatures and logic, we state three hypotheses.

Expected Course Grades and SET

The importance of raising one’s SET scores is clear, given the significant role that these scores play in retention and promotion decisions for faculty. Moreover, as dozens of studies show, a positive and moderate association between expected course grades and SET scores exists, ranging from .10 on the low end to .47 on the high end (Bacon & Novotny, 2002; Spooren et al., 2013). Because of this association, some instructors attempt to influence SET scores by awarding higher expected course grades (hereafter referred to as “grades” for brevity), a practice reported by 24% of instructors in one study (Simpson & Siguaw, 2000). Such faculty might not feel a need to award artificially high grades if they knew how students’ perceptions of justice might influence this relationship. While the literature has shown the direct effect of some justice perceptions on SET, it has not shown how to attenuate the effect of grades on SET, which is what the current study sets out to do.

Justice Perceptions

One reason for this association between grades and SET is that many students think of grades in terms of distributive justice (e.g., Wendorf & Alexander, 2005). Research conducted across various organizational contexts shows that people who are treated unfairly sometimes try to restore justice through revenge against their perpetrators (Adams, 1965; Bies & Tripp, 1996; Hershcovis et al., 2007; Skarlicki & Folger, 1997; Tripp, Bies, & Aquino, 2007). The revenge motive may partially explain why studies show that when students perceive that grades they received are unfair, they give lower SET. Much like disgruntled employees, disgruntled students might seek to get even as a way to restore an injustice perceived due to receiving a low grade, using lower SET as an easy, convenient, and anonymous punishment of their instructor (Chory-Assad, 2002; Colquitt, 2001; Marsh & Roche, 2000; Spooren et al., 2013; Tata, 1999). Chory-Assad and Paulsel (2004) indeed found that perceptions of poor distributive justice do correlate with in-class behaviors such as revenge toward the instructor, student hostility, and aggression. Clayson, Frost, and Sheffet (2006) also conclude that students might reciprocate (i.e., get even) for low grades by giving low SET and reward high grades with high SET. Even after controlling for rival explanations, such as student performance (higher performance can cause both higher grades and higher SET), Clayson et al. found that the grades-SET relationship still exists.

Mediating Effect of Distributive Justice Perception

We posit that—(a) because grades are an important outcome of courses for students, (b) because people often compare important outcomes with some distributive justice referent (Adams, 1965; Folger, 1986), and (c) because distributive injustice increases revenge (Adams, 1965; Bies & Tripp, 1996)—that therefore distributive justice perception is the mediator of the relationship between grades and SET. However, it is not established in the literature that distributive justice perception mediates the relationship between grades and SET. Mediation can be inferred from extant research that shows that grades predict distributive justice perception (e.g., Cherry, Ordonez, & Gilliland, 2003; Gordon & Fay, 2010), and that distributive justice perception affects SET (e.g., Colquitt, 2001; Lersch & Greek, 2001). Thus, it could be that grades do not necessarily directly affect SET, but this effect is achieved through altered distributive justice perception. To our knowledge, however, no study has actually tested this mediation, though Wendorf and Alexander (2005) come close by separately measuring both the grades-to-distributive-justice and distributive-justice-to-SET relationships. It is important to directly test for mediation to demonstrate that grades influence SET because students think of grades in justice terms. Thus, we hypothesize the following:

Hypothesis 1: Distributive justice perception mediates the relationship between grades and SET.

Procedural Justice Perception and the Fair Process Effect

While distributive justice focuses on the favorability of outcomes, procedural justice is concerned with decision-making processes used to determine those outcomes (Thibaut & Walker, 1978). For example, a decision process will generally be perceived as fair if rules are applied consistently, members have voice in the decision-making process, and one can appeal, should the decision prove to be flawed (Leventhal, 1980). It is well known in the management literature that the perception of fair processes not only directly improves employees’ reactions to decisions but it also attenuates the influence of distributive justice perceptions on employees’ reactions to and acceptance of decisions (Brockner & Wiesenfeld, 1996). In other words, recipients tend to accept decisions that they perceive are procedurally fair, regardless of distributive justice, and it is only when both the outcome and the process are perceived to be unfair do recipients not accept decisions and react negatively. This pattern of results is known as the fair process effect. Van den Bos (2005) claims that the fair process effect is “arguably the most replicated and robust finding in the literature on organizational justice and one of the most frequently observed phenomena and among the basic principles in the organizational behavior and management literatures” (p. 274). Organizational scholars generally consider this moderating role of the fair process effect crucial to managing human behavior (e.g., Brockner, 2006; Cropanzano, Bowen, & Gilliland, 2007), because it means that decision makers need not worry about the perceived fairness of outcomes they distribute, as long as they use transparently fair processes.

The literature suggests several theoretical explanations for how the fair process effect unfolds (Brockner & Wiesenfeld, 1996; Van den Bos, 2005). First, the instrumental explanation (Thibaut & Walker, 1978) proposes that when people have process control, their need for decision control lessens. Applied to the classroom, this explanation suggests that students who know what the course procedures are (e.g., how work is graded), and thus know what the instructor wants, have more control over their grades because they have the information they need to succeed.

Second, referent cognitions theory (Folger, 1986) suggests that fair processes make it difficult for recipients to imagine how the decision outcome might have turned out differently than it actually turned out, and thus the outcome seems more justified. That is, as long as one knows that an elaborate set of fair procedures were used, then one can predict what the outcome should be based on one’s efforts or performance. When the outcome received matches the expected, referent outcome, then the recipient does not have a justifiable reason to complain about the outcome, and so accepts the outcome. Based on this theory, we expect that the students who perform badly and get lower grades, but who also know what procedures and criteria the instructor uses to grade them, cannot imagine how their grades might have turned out differently. For instance, students who put in minimum work, and who can see that their assignment does not meet the minimums established in a grading rubric, cannot imagine how they might have been awarded an “A” grade, and thus should be less upset by their grade.

Third, the relational model of authority (Tyler & Lind, 1992) proposes that fairness primarily has value because it indicates recipients’ standing with their authority figures (e.g., bosses, top management, and instructors). Procedural justice is proposed to be a better indicator of this standing in part because it is relatively enduring and stable compared with perceptions of distributive justice, which are usually tied to a single outcome. This theory allows us to speculate that when students see the rigorously fair processes that the instructor designed into the course, then they may believe that the instructor genuinely cares about students, and respects their standing in the university.

Fourth, the uncertainty management explanation (Lind, 2001; Van den Bos, 2001) states that when recipients are uncertain about outcome information, they rely on process information to make judgments. That is, when a recipient does not have enough information to make equity comparisons (e.g., an employee who cannot determine if one’s salary is equitable because the employee does not know other employees’ salaries), then the recipient will examine process information. When applied to a classroom setting, this theory suggests that students who receive low grades, and who also do not know other students’ grades, are not left without any information to form judgments; rather, they use available process information to make judgments. For these reasons, students may rely on process information, when it is available, so much so that the outcome information will matter much less. Outcome information (e.g., knowing one received one of the lowest grades in class) may matter mainly when process information is lacking.

Although the fair process effect in the classroom has a solid theoretical foundation, empirical evidence for its effectiveness in the classroom context is limited and mixed. Some studies have demonstrated that perceptions of procedural justice have a main effect on SET, but did not find a moderating effect of procedural justice perception on grades on SET or distributive justice on SET (e.g., Duplaga & Astani, 2010; Rodabaugh & Kravitz, 1994; Tyler & Caine, 1981; Wendorf & Alexander, 2005). For instance, Chory-Assad and Paulsel (2004) examined how the fair process effect might explain why controlling for procedural justice perception made the distributive justice perception correlations with hostility and aggression disappear, and they found no significant interaction effect in their study. Additionally, Tata (1999) explored the moderating effect of procedural justice by conducting an experimental scenario study, where he experimentally manipulated the fairness of grading procedures (fair or unfair) and outcomes (expected grade or lower than expected grade) to assess how students would ascribe SET evaluations for the hypothetical instructors in each experimental condition. Tata (1999) found the fair process effect: only when both the outcome and process were unfair did hypothetical SET scores drop. However, despite demonstrating the moderating effect of procedural justice, Tata’s study should not stand alone, given its reliance on experimental scenarios, rather than on actual class experiences. Therefore, to provide empirical evidence that can help instructors realize they can avoid receiving low SET scores after assigning grades their students deserve, Tata’s (1999) scenario study requires a field replication that relies on students’ evaluations of actual instructors and classroom experiences. Thus, we propose and test the following hypothesis:

Hypothesis 2: Procedural justice perception moderates the relationship between distributive justice perception and SET, such that the positive relationship between distributive justice perception and SET becomes weaker when procedural justice perception is high, compared with when procedural justice perception is low.

Taken together, our hypotheses suggest that the mediating role of distributive justice perception in the relationship between grades and SET depends on the level of procedural justice perception. From a statistical perceptive (Edwards & Lambert, 2007; Preacher, Rucker, & Hayes, 2007), our model presents a case of moderated mediation. Specifically, the amount to which the mediator (i.e., distributive justice perception) translates the effect of the predictor (i.e., grades) on the outcome (i.e., SET) may be contingent on the level of the moderator (i.e., procedural justice perception), as depicted in Figure 1. Specifically, we propose the following hypothesis:

Hypothesis 3: The relationship between grades and SET via distributive justice perception becomes weaker when procedural justice perception is high, compared with when procedural justice perception is low.

Method

Participants

Participants were recruited from three, large public universities located in different geographical areas of the United States (Pacific Northwest, Southwest, and Midwest, 34.7%, 29.1%, and 36.2% of the total sample, respectively). Participation in this online study was offered to undergraduate and graduate students in select business and psychology courses as a course extra-credit opportunity. After the elimination of incomplete surveys, and multiple surveys completed by the same student, data from 250 students were included in the analyses of our hypotheses.¹ Participant characteristics include a mean age of 23.70 (SD = 6.23) with the majority of participants self-reporting as women (63%) and Caucasian (81%). Also, 93% were full-time students and 7% part-time; 16% lived on campus, while 81% lived off-campus. Finally, 95% were undergraduate students, while 5% were graduate students. Note that these percentages are consistent with the populations from which they were sampled.

Measures

Student Evaluation of Teaching

SET items used in this study were designed to most closely correspond to the actual SET items used by universities where the data were collected, in order to make the study as generalizable as possible to actual SET ratings. Those exact four items are (a) “Overall, this was an informative class. I learned a great deal,” (b) “All things considered, the instructor was a good instructor,” (c) “I would recommend this course and instructor to another student,” and (d) “I would take another course by this instructor.” Responses to items were assessed on a 7-point Likert-type scale with anchors ranging from 1 (strongly disagree) to 7 (strongly agree).

Distributive Justice Perception

Participants’ perception of distributive justice was assessed with a set of six items from the Chory-Assad and Paulsel (2004) classroom justice scale. Participants were asked to compare the grade they actually received in the whole course with the following reference points: (a) other students’ grades, (b) their own expected grade, (c) grade they felt they deserved to receive, (d) grades received in similar courses, (e) grades that other students would receive in the course, and (f) grade on their last exam or assignment in that class. Responses to items were assessed on a 7-point Likert-type scale with anchors ranging from 1 (extremely unfair) to 7 (extremely fair).

Procedural Justice Perception

Participants’ perception of procedural justice was captured with a set of 15 items from the Chory-Assad and Paulsel (2004) classroom justice scale, reflecting key teaching practices that instructors typically perform. For example, students were asked to rate the fairness of missed-work make-up policies, course attendance policies, grading scale for the course, schedule of exams, and instructor’s expectations. The scale also included items that assessed course issues usually not addressed on course syllabi, such as the quality of questions on exams, and how well the instructor tracks participation. Furthermore, we added seven additional items to better reflect Leventhal’s (1980) list of features of fair processes (e.g., for consistency of rule application, one item read, “The way the instructor followed her or his syllabus was”). Responses to items were assessed on a 7-point Likert-type scale with anchors ranging from 1 (extremely unfair) to 7 (extremely fair), with a “nonapplicable (NA)” option, which was replaced with the actual scale mean based on other completed items.² See the appendix for a list of all items.

Grade

On completion of the attitudinal assessments described above, participants reported the grade that they received for the course. A standard grading scheme was used (i.e., letter grades ranging from “A” to “F” with plus and minus designations), which we converted to a numerical grade-point scale ranging from 0 to 4, corresponding to grades F, D, C, B and A, for analysis.

Qualitative Assessment of Fairness

Given the importance of distributive and procedural justices, we sought to qualitatively explore what fairness issues occur to students when asked to recall a course. In the survey, before participants responded to the various scales on perceived fairness, they answered an open-ended question: “All things considered in the course, was the instructor fair? Why or why not?”

Procedure

In order to encourage variance in grades, participants were randomly assigned to survey conditions where they were asked to pick an actual course from the previous semester in which they received either the highest grade (n = 133) or the lowest grade (n = 117). Once assigned to that condition, participants were asked to answer the survey as described in the measures section related to their perceptions of fairness of only that one course.

As a check of the random assignment method, we conducted t tests and found that students in the highest grade condition reported higher grades (M = 3.59, SD = 0.64) compared with those in the lowest grade condition (M = 2.19, SD = 1.16), t(246) = 11.96, p < .001, Cohen’s d = 1.49, on a grade-point scale of 0.0 to 4.0. Moreover, we found that students in the highest grade condition reported higher distributive justice perception (M = 5.89, SD = 0.95) than those in the lowest grade condition (M = 4.70, SD = 1.45), t(248) = 7.80, p < .001, Cohen’s d = .97.

Analytic Strategies

Two interlinked steps were used to test our hypotheses. First, we examined a simple mediation model (Hypothesis 1) using a SPSS macro (Model 4) developed by Hayes (2013). Next, we included procedural justice perception as a moderator in the model to test Hypothesis 2 and Hypothesis 3. To test the moderation in Hypothesis 2 and the moderated mediation in Hypothesis 3, we used the same Hayes’ SPSS macro (Model 14; Preacher et al., 2007; see a similar method in Cole, Walter & Bruch, 2008). This macro, using bootstrapping methodology, tests the significance of the mediation effect at different values of the moderator.

Results

Table 1 presents descriptive statistics, Cronbach’s alpha, and zero-order product-moment correlations among the variables of interest. Student age and gender as well as instructor age, gender, and race were not significantly related to distributive justice perception or SET. Based on Becker’s (2005) recommendation that the inclusion of unnecessary controls reduces statistical power and yields biased estimates, student age and gender as well as instructor age, gender, and race were excluded from any further analyses. However, we included the high- versus low-grade condition as a control variable in the following analyses.

Table 1.

Descriptive Statistics, Scale Reliabilities, and Correlations.

Variable	N	M	SD	1	2	3	4	5	6	7	8	9
1. Student evaluation of teaching	249	5.20	1.90	(.96)
2. Distributive justice of course	250	5.32	1.35	.56**	(.94)
3. Procedural justice	250	5.61	1.14	.68**	.75**	(.97)
4. Grades	248	2.94	1.15	.50**	.57**	.48**
5. Student age	246	23.70	6.22	.04	.05	.08	.16*
6. Student gender	247	1.64	0.48	−.05	−.03	−.04	−.01	.03
7. Grade condition	250	1.47	0.50	−.39**	−.44**	−.39**	−.61**	−.03	−.00
8. Instructor age	246	5.47	2.17	−.06	.05	.03	.10	.06	.05	.04
9. Instructor sex	247	0.33	0.47	−.03	−.08	−.08	−.07	−.15*	−.05	−.00	−.20**
10. Instructor race	250	0.17	0.38	−.10	−.12	−.20**	−.18**	−.02	.08	.12	−.01	.14*

Note. Student gender: 1 = male, 2 = female; Grade condition: 1 = high grade, 2 = low grade; Instructor gender: 0 = male, 1 = female; Instructor race: 0 = White, 1 = non-White. Coefficient alpha reliabilities are on the diagonal in parentheses.

p < .05. **p < .01.

Confirmatory Factor Analyses

Before examining the discriminant validity of our three perceptual scales (i.e., SET, distributive justice perception, procedural justice perception), we followed Little, Cunningham, Shahar, and Widaman’s (2002) recommendations to create item parcels. Three item-parcels were created by sequentially assigning items per parcel based on the highest to lowest item-to-construct loadings/correlations. Subsequent confirmatory factor analyses were conducted with Mplus 7.0 (Muthén & Muthén, 2010) using maximum likelihood estimation. The hypothesized three-factor model, χ²(24) = 24.95, standardized root mean square residual (SRMR) = 0.02, root mean square error of approximation (RMSEA) = 0.01, comparative fit index (CFI) = 1.00, displayed an excellent fit to the data, and fit the data significantly better than a two-factor model where distributive justice perception and procedural justice perception were combined (SRMR = 0.06, RMSEA = 0.20, CFI = 0.86), with a significant reduction in chi square of 269.71(Δdf = 2, p < .01). This three-factor model also fit the data significantly better than a one-factor model (SRMR = 0.12, RMSEA = 0.33, CFI = 0.61), with a significant reduction in chi square of 737.64 (Δdf = 3, p < .01). Therefore, CFA provided support for the discriminant validity of the three perceptual scales. Additionally, average variance extracted values (Fornell & Larcker, 1981) for SET, distributive justice perception, and procedural justice perception are .91, .88, and .91, respectively.

Tests of Mediation

Table 2 presents the results for Hypotheses 1. First, replicating previous studies, grades were positively related to SET (Β = 0.68, p < .01). After including distributive justice perception, although there was still a significant relationship between grades and SET, the coefficient was much smaller (Β = 0.37, p < .01). Grades were positively associated with distributive justice perception, as indicated by a significant unstandardized regression coefficient (Β = 0.56, p < .01). Furthermore, the positive relationship between distributive justice perception and SET, controlling for grades, was supported (Β = 0.57, p < .01). Finally, grades were found to have a mediated, positive effect on SET through distributive justice perception (0.31), which supported Hypothesis 1. A 95% bias-corrected bootstrap confidence around the mediated effect was entirely above zero (confidence interval [.20, .47]). Thus, Hypotheses 1 received support. Grades, distributive justice perception, and the grade condition together explained 37% of variance in SET.

Table 2.

Regression Results for Simple Mediation.

Variable	B	SE	β	t	p
Direct and total effects
SET regressed on grades, controlling for condition	0.68	0.11	.41	9.05	<.001
Distributive justice regressed on grades, controlling for condition	0.56	0.07	.47	7.33	<.001
SET regressed on distributive justice, controlling for grades and condition	0.57	0.09	.40	6.32	<.001
SET regressed on grades, controlling for distributive justice and condition	0.37	0.12	.22	3.15	<.001
	M	SE	β	LL 95% CI	UL 95% CI
Bootstrap results for indirect effect
Effect	0.31	0.07	.16	.20	.47

Note. N = 247; Bootstrap sample size = 5,000. Condition: 1 = high grade; 2 = low grade; SET = student evaluation of teaching; CI = confidence interval; LL = lower limit; UL = upper limit. Unstandardized regression coefficients are reported.

Tests of Moderation and Moderated Mediation

Hypothesis 2 predicted that the positive relationship between distributive justice perception and SET would be stronger for students who perceived low procedural justice than for those who perceived high procedural justice. A stepwise regression indicated that the cross-product term between distributive justice perception and procedural justice perception on SET was significant (Β = −.12, p = .02), after controlling for the grade condition. We also plotted simple slopes (see Figure 2) at 1 standard deviation above and below the mean of procedural justice perception. As can be seen in Figure 2, the form of the interactions was consistent with our prediction. Specifically, there was no significant relationship between distributive justice perception and SET when perceptions of procedural justice were high (simple slope = .01, t = .11, p = .96). However, when procedural justice perception was low, distributive justice perception was positively related to SET (simple slope = .30, t = 2.56, p < .01), supporting the fair process effect (i.e., Hypothesis 2).

To test Hypothesis 3, we examined the conditional indirect effect of grades on SET (through distributive justice perception) at three values of procedural justice perception: the mean, 1 standard deviation above the mean, and 1 standard deviation below the mean. It was indicated that one of the three conditional indirect effects (based on the moderator value at −1 SD) was positive and significantly different from zero (the conditional indirect effect was 0.14). On the other hand, when the procedural justice perception is high (based on the moderator values at the mean and +1 SD), the conditional indirect effect was not significant. Bootstrapped confidence intervals corroborated these results. Thus, Hypothesis 3 was supported, such that the indirect and positive effect of grades on SET through distributive justice perception was observed when levels of procedural justice perception were low, but not when procedural justice perception was moderate to high (for a summary of the significant pathways, please see Figure 3).

Exploratory Analysis of Qualitative Data

Given the importance of distributive and procedural justice perceptions, we explored what fairness issues occur to students when asked to recall a course but before being primed by our scale items. Inductive qualitative coding techniques (e.g., Glaser & Strauss, 1967; Krippendorff, 2004; Patton, 2001) were used to analyze the content of participant responses. Each of the participant answers was read by a primary researcher several times to identify themes that served as coding categories. After the initial reading of responses to identify themes, another primary researcher read through responses to edit and validate themes. Across 249 participant responses, 26 themes were identified. Next, two trained undergraduate research assistants coded each of the participant responses into the themes designated for each question. The research assistants independently coded the participant responses. The interrater agreement between research assistants was 95%. A primary researcher reviewed all coding discrepancies and made the final coding decision.

The most frequent (i.e., in more than 5% of participants’ responses) themes were (a) the instructor graded fairly or unfairly (36%), (b) whether the instructor treated everyone equally or was biased toward some students (17%), (c) the instructor was helpful with learning (16%), (d) the appropriateness of rigor level (15%), (e) the instructor provided study guides (11%), (f) exam items did not match material covered (9%), (g) the instructor used rubrics that matched criteria/expectations (10%), (h) the instructor followed a clear syllabus (6%), (i) the instructor took feedback from students (6%), and (j) the instructor provided policies for make-up work or absences (5%).

Discussion

Our study provides support for Hypotheses 1 to 3. Consistent with Hypothesis 1, distributive justice perception partially mediated the relationship between grades and SET. Moreover, consistent with Hypothesis 2, procedural justice perception moderated the relationship between distributive justice perception and SET. Finally, in support of Hypothesis 3, procedural justice perception moderated the indirect relationship between grades and SET through distributive justice perception, such that only when procedural justice perception was low did grades predict SET via distributive justice perception. This pattern aligns with what is predicted by the fair process effect. Thus, the study provides evidence that fair process effect generalizes to the classroom setting.

The exploratory analysis of participants’ open-ended responses, which participants completed before being exposed to our specific fairness scales, demonstrated their concerns with how they were graded by instructors. Some of their concerns resemble Leventhal’s (1980) procedural justice principles of bias suppression (e.g., treating students equally) and consistency of rule application (e.g., followed the syllabus, used rubrics that matched official criteria). Two other relatively major concerns were that the instructor helps students succeed (e.g., provided study guides) and that the level of rigor is appropriate.

Practical Implications: Designing Fair Courses

Many instructors fear that if they issue low grades, then their students will punish them by giving low SET scores. We reviewed the literature and conducted a study to test whether this concern can be mitigated. Our literature review notes that many studies have found a correlation between grades and SET, and some suggest that distributive justice perception may mediate this effect. Our results support this suggestion. Our results also support that perceptions of fair processes moderates this effect such that when students perceive high fair process, the effects of grades and distributive justice perception on SET are attenuated (and in our sample, eliminated). These findings collectively suggest that instructors should not worry so much about assigning the grades students deserve, even if that means giving out low grades, as long as the instructors use perceived fair processes. As such, the fair process effect can attenuate the relationship between distributive justice perception and SET (see Figure 3).

The importance of fair process requires a discussion of what specific factors students perceive as fair classroom procedures. Leventhal (1980) argues that when it comes to process, individuals believe that fair decision-making processes should have the following features: (a) consistency (i.e., rules are applied consistently across different people and are stable over time); (b) representativeness (i.e., the process reflects the interests of all affected subgroups); (c) accuracy of information (i.e., information on which decisions are based should be correct and relevant); (d) bias suppression (i.e., the decision makers should minimize conflicts of interest); and (e) correctability (i.e., bad decisions can be reversed or remade). In addition, perceptions of procedural justice are also enhanced when an individual is given voice into decision-making process (Folger, 1977).

The results of our study reveal the following criteria for enacting fair classroom practices, which closely correspond to Leventhal’s five criteria listed above. Namely, instructors should: (a) follow the course rules by using grading rubrics that match stated criteria, and by aligning their course presentation and expectations to the syllabus; (b) incorporate students’ interests and voice by taking feedback from students; (c) be aware of the fact that instructors might succumb to biases they have for or against particular students, and then avoid these biases by grading blindly; and, (d) correct grades by providing policies for make-up work and absences. Other studies have found other specific procedural justice concerns that students have, which align with Leventhal’s criteria. For example, when Horan, Chory, and Goodboy (2010) surveyed students to determine what behaviors they considered unfair, the most frequent procedural violation, reported by 33% of students, was inconsistent grading. Also, Houston and Bettencourt (1999) used the critical-incident method to uncover those behaviors students perceive as fair and unfair. They found that when students can appeal their course grades and when course policies are consistently enforced, students’ perceptions of fairness increase.

Based on ours and others’ findings, we recommend instructors enact the following procedures to create a fair classroom. First, instructors should make clear what is graded and how it is graded. For instance, instructors should use grading rubrics, use them consistently, and share them with students. According to Jonsson and Svingby (2007) who reviewed 75 empirical studies on grading rubrics, good grading rubrics are analytic (i.e., rating specific dimensions rather than holistically rating the entire assignment), topic-specific, contain exemplars, and raters are trained in their use. In a more recent review, Panadero and Jonsson (2013) found that well-designed rubrics are those that are so transparent as to make assessment criteria explicit and easy to understand.

Second, all course policies, such as late assignment submissions, should be stated in writing (e.g., in the syllabus) and then followed closely, so that students can see that policies are being followed consistently. Good syllabi should be detailed (Davis, 2009; Slattery & Carlson, 2005), contain information pertaining to content and learning goals, including assignments’ roles in learning goals, and outline how class activities or assignments will be graded (Slattery & Carlson, 2005).

Third, if possible, instructors should include grade-appeal procedures in their course policies. Kravitz, Stone-Romero, and Ryer (1997) found that students more favorably evaluated grade-appeal procedures that follow Leventhal’s rules of procedural justice. Thus, if possible, instructors could have students write their appeals and submit their appeals by their student ID numbers rather than by their names. This would allow the instructors to suppress their own biases, give the students voice, and perhaps increase accuracy of information by reevaluating the original student work. Should a grade appeal move up to a panel, then the panel could include students, which would increase representativeness.

Adding such processes to a course may seem like a lot of work, or at least perhaps more work than just grading leniently. However, we believe that instituting fair processes is the superior option, for several reasons. One, rampant use of grading leniency may contribute to grade inflation, which already is advancing at a rate of about .14 point per decade (Rojstaczer, 2003). Two, by grading leniently, instructors reduce variance in grades, thereby diminishing the power of grades to extrinsically motivate student to work harder. Three, by grading leniently, instructors actually may reduce perceptions of distributive justice as grades become more inequitable (i.e., awarding the same grades for varying effort and quality of work), which we show are associated with lower SET. In fact, by ignoring procedural justice and grading leniently, instructors risk ending up creating perceptions of both unfair outcomes and unfair process, a deadly combination that is associated with lower SET and, at least in the workplace, leads to complaints (Van den Bos, 2005).

Limitations

Although we attempted to design a methodologically sound study, we would like to note limitations inherent with our research. Primarily, the experimental method of our data limits the external validity of our findings in several ways. First, the students were asked to rate an instructor 2 to 3 months after the term was over. This procedure differs from real SET because in our study participants considered the final course grades they actually received, whereas when students complete real SET, they base their evaluation on the final course grade they expect to receive. Thus, our study does not capture the effects of the uncertainty students may feel regarding their grades or that may cause them to misestimate their final grades. However, the final-grade uncertainty students experience at the end of the semester may be less pronounced in today’s academic environment where student grades are readily and continuously available via online learning platforms such as Blackboard or Canvas. In such an environment, the only grade uncertainty students may experience is how they might perform on their final exams.

Second, because the study was retrospective, students may have not had a good memory of certain course procedures 2 to 3 months later, and their overall evaluation of the instructor and course may vary from what they thought when the course was still in session. This is likely if cognitive dissonance distorts memory such that students remember mostly those details that are consistent with the final grade—perhaps especially so if the final grade was a negative surprise due to misestimation, creating a “sour grapes” memory of course details; or if the final grade was a positive surprise, creating a rosier memory of the course. Memory distortion also may occur due to the “peak-end rule” of memory where the peak event and last event in an episode (e.g., a difficult final exam, a lower than expected course grade) bias one’s evaluation of the whole episode (Kahneman, Fredrickson, Schreiber, & Redelmeier, 1993). If correct, the peak-end rule may partially explain why our study found a grades-SET correlation, at .50, at the high end of typical grades-SET correlations found in more naturalistic settings.

Third, because we asked students to recall the course with either the highest or lowest grade, we likely inflated variance in the grades independent variable. Such higher variance may inflate correlations in the study. This may be another reason why our study found such a high grades-SET correlation. Note, however, this aspect of the research design creates a conservative test for the fair process effect. That is, we find that in courses where students strongly react to grades when completing SET, even then fair processes can strongly attenuate the influence of grades on SET.

Fourth, although we sampled multiple undergraduate majors and graduate students, the nature of the data we collected—that is, not knowing which majors students were pursuing, and collecting too small a sample of graduate students—prevented us from breaking out results by major or undergraduate/graduate status. While this may make our findings more broadly generalizable across a university, it makes them less generalizable to a business school. Also, given known differences among different student demographics regarding, for example, grading leniency (e.g., Bacon & Novotny, 2002), this may limit our statistical power by not controlling for these variables.

One final limitation of the study is that the data are derived from a self-report and cross-sectional data collection. A multiwave design would reduce common-method bias and would potentially allow inferences of causality.

Conclusion

Many instructors have long assumed that giving low grades would reduce students’ evaluations of their courses. However, in this study, we provide evidence that this fear of repercussions following low grades may be exaggerated. We found that the fair process effect—which is well known in the management literature to preempt complaints from subordinates who are denied the outcomes they believe they deserve—can also lower the likelihood that students who are denied the grades they believe they deserve will give instructors poor evaluations. Therefore, as long as instructors use transparently fair procedures in their courses, they need not fear the grades-SET association, and therefore they need not react superficially to pressure for maintaining high teaching evaluations, such as by grading more leniently. Instead, instructors may confidently give students the grades they deserve.

Footnotes

Appendix

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Notes

References

Adams

J. S.

(1965). Inequity in social exchange. In Berkowitz

(Ed.), Advances in experimental social psychology (Vol. 2, pp. 267-299). New York, NY: Academic Press.

Bacon

D. R.

Novotny

(2002). Exploring achievement striving as a moderator of the grading leniency effect. Journal of Marketing Education, 24, 4-14.

Becker

T. E.

(2005). Potential problems in the statistical control of variables in organizational research: A qualitative analysis with recommendations. Organizational Research Methods, 8, 274-289.

Bies

R. J.

Tripp

T. M.

(1996). Beyond distrust: “Getting even” and the need for revenge. In Kramer

R. M.

Tyler

(Eds.), Trust in organizations (pp. 246-260). Thousand Oaks, CA: Sage.

Brockner

(2006). Why it’s so hard to be fair. Harvard business review, 84(3), 122-129.

Brockner

Wiesenfeld

B. M.

(1996). An integrative framework for explaining reactions to decisions: Interactive effects of outcomes and procedures. Psychological Bulletin, 120, 189-208.

Cherry

Ordonez

L. D.

Gilliland

S. W.

(2003). Grade expectations: The effects of expectations on fairness and satisfaction perceptions. Journal of Behavioral Decision Making, 16, 375-395.

Chory-Assad

R. M.

(2002). Classroom justice: Perceptions of fairness as a predictor of student motivation, learning, and aggression. Communication Quarterly, 50, 58-77.

Chory-Assad

R. M.

Paulsel

M. L.

(2004). Classroom justice: Student aggression and resistance as reactions to perceived unfairness. Communication Education, 53, 253-273.

10.

Clayson

D. E.

(2009). Student evaluations of teaching: Are they related to what students learn? A meta-analysis and review of the literature. Journal of Marketing Education, 31, 16-30.

11.

Clayson

D. E.

Frost

T. F.

Sheffet

M. J.

(2006). Grades and the student evaluation of instruction: A test of the reciprocity effect. Academy of Management Learning & Education, 5, 52-65.

12.

Cole

M. S.

Walter

Bruch

(2008). Affective mechanisms linking dysfunctional behavior to performance in work teams: A moderated mediation study. Journal of Applied Psychology, 93, 945-958.

13.

Colquitt

J. A.

(2001). On the dimensionality of organizational justice: A construct validation of a measure. Journal of Applied Psychology, 86, 386-400.

14.

Cropanzano

Bowen

D. E.

Gilliland

S. W.

(2007). The management of organizational justice. Academy of Management Perspectives, 21, 34-48.

15.

Crumbley

D. L.

Flinn

R. E.

Reichelt

K. J.

(2010). What is ethical about grade inflation and coursework deflation? Journal of Academic Ethics, 8, 187-197.

16.

Davis

B. G.

(2009). Tools for teaching. Hoboken, NJ: John Wiley.

17.

Deutsch

(1973). The resolution of conflict: Constructive and destructive processes. New Haven, CT: Yale University Press.

18.

Duplaga

E. A.

Astani

(2010). An exploratory study of student perceptions of which classroom policies are fairest. Decision Sciences Journal of Innovative Education, 8, 9-33.

19.

Edwards

J. R.

Lambert

L. S.

(2007). Methods for integrating moderation and mediation: a general analytical framework using moderated path analysis. Psychological methods, 12(1), 1.

20.

Folger

(1977). Distributive and procedural justice: Combined impact of voice and improvement on experienced inequity. Journal of Personality and Social Psychology, 35, 108-119.

21.

Folger

(1986). A referent cognitions theory of relative deprivation. In Olson

J. M.

Herman

C. P.

Zanna

M. P.

(Eds.), Relative deprivation and social comparison: The Ontario symposium (Vol. 4, pp. 33-55). Hillsdale, NJ: Lawrence Erlbaum.

22.

Folger

Rosenfield

Grove

Corkran

(1979). Effects of “voice” and peer opinions on responses to inequity. Journal of Personality and Social Psychology, 37, 2253-2261.

23.

Fornell

Larcker

D. F.

(1981). Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research, 18, 39-50.

24.

Glaser

B. G.

Strauss

A. L.

(1967). The discovery of grounded theory: Strategies for qualitative research. Chicago, IL: Aldine.

25.

Gordon

M. E.

Fay

C. H.

(2010). The effects of grading and teaching practices on students’ perceptions of grading fairness. College Teaching, 58, 93-98.

26.

Greenwald

A. G.

Gillmore

G. M.

(1997). No pain, no gain? The importance of measuring course workload in student ratings of instruction. Journal of Educational Psychology, 89, 743-751.

27.

Hayes

A. F.

(2013). PROCESS SPSS Macro [Computer software and manual].

28.

Hershcovis

M. S.

Turner

Barling

Arnold

K. A.

Dupre

K. E.

Inness

. . . Sivanathan

(2007). Predicting workplace aggression: A meta-analysis. Journal of Applied Psychology, 92, 228-238.

29.

Horan

S. M.

Chory

R. M.

Goodboy

A. K.

(2010). Understanding students’ classroom justice experiences and responses. Communication Education, 59, 453-474.

30.

Houston

M. B.

Bettencourt

L. A.

(1999). But that’s not fair! An exploratory study of student perceptions of instructor fairness. Journal of Marketing Education, 21, 84-96.

31.

Jonsson

Svingby

(2007). The use of scoring rubrics: Reliability, validity and educational consequences. Educational Research Review, 2, 130-144.

32.

Kahneman

Fredrickson

B. L.

Schreiber

C. A.

Redelmeier

D. A.

(1993). When more pain is preferred to less: Adding a better end. Psychological science, 4, 401-405.

33.

Kravitz

D. A.

Stone-Romero

E. F.

Ryer

J. A.

(1997). Student evaluations of grade appeal procedures: The importance of procedural justice. Research in Higher Education, 38, 699-726.

34.

Krippendorff

(2004). Content analysis: An introduction to its methodology. London, England: Sage.

35.

Lersch

K. M.

Greek

(2001). Exploring the beliefs surrounding student evaluations of instruction in criminology and criminal justice undergraduate courses. Journal of Criminal Justice Education, 12, 283-299.

36.

Leventhal

G. S.

(1980). What should be done with equity theory? New approaches to the study of fairness in social relationships. In Gergen

Greenberg

Willis

(Eds.), Social exchange: Advances in theory and research (pp. 27-55). New York, NY: Plenum Press.

37.

Lind

E. A.

(2001). Fairness heuristic theory: Justice judgments as pivotal cognitions in organizational relations. Advances in organizational justice, 56(8).

38.

Little

T. D.

Cunningham

W. A.

Shahar

Widaman

K. F.

(2002). To parcel or not to parcel: Exploring the question and weighing the merits. Structural Equation Modeling, 9, 151-173.

39.

Marsh

H. W.

Roche

L. A.

(2000). Effects of grading leniency and low workload on students’ evaluations of teaching: Popular myth, bias, validity, or innocent bystanders? Journal of Educational Psychology, 92, 202-228.

40.

Muthén

L. K.

Muthén

B. O.

(2010). Mplus user’s guide (5th ed). Los Angeles, CA: Muthén & Muthén.

41.

Panadero

Jonsson

(2013). The use of scoring rubrics for formative assessment purposes revisited: A review. Educational Research Review, 9, 129-144.

42.

Patton

M. Q.

(2001). Qualitative evaluation and research methods (3rd ed.). Thousand Oaks, CA: Sage.

43.

Preacher

K. J.

Rucker

D. D.

Hayes

A. F.

(2007). Addressing moderated mediation hypotheses: Theory, methods, and prescriptions. Multivariate Behavioral Research, 42, 185-227.

44.

Rodabaugh

R. C.

Kravitz

D. A.

(1994). Effects of procedural fairness on student judgments of professors. Journal on Excellence in College Teaching, 5, 67-83.

45.

Rojstaczer

(2003). Grade inflation at American colleges and universities. Retrieved from http://www.gradeinflation.com/

46.

Simpson

P. M.

Siguaw

J. A.

(2000). Student evaluations of teaching: An exploratory study of the faculty response. Journal of Marketing Education, 22, 199-213.

47.

Skarlicki

D. P.

Folger

(1997). Retaliation in the workplace: The roles of distributive, procedural, and interactional justice. Journal of Applied Psychology, 82, 434-443.

48.

Slattery

J. M.

Carlson

J. F.

(2005). Preparing an effective syllabus: Current best practices. College Teaching, 53, 159-164.

49.

Spooren

Brockx

Mortelmans

(2013). On the validity of student evaluation of teaching the state of the art. Review of Educational Research, 83, 598-642.

50.

Tata

(1999). Grade distributions, grading procedures, and students’ evaluations of instructors: A justice perspective. Journal of Psychology, 133, 263-271.

51.

Thibaut

Walker

(1978). Theory of procedure. California Law Review, 66, 541-566.

52.

Tripp

T. M.

Bies

R. J.

Aquino

(2007). A vigilante model of justice: Revenge, reconciliation, forgiveness, and avoidance. Social Justice Research, 20, 10-34.

53.

Tyler

T. R.

Lind

E. A.

(1992). A relational model of authority in groups. In Advances in experimental social psychology (Vol. 25, pp. 115-191). Academic Press.

54.

Tyler

T. R.

Caine

(1981). The influence of outcomes and procedures on satisfaction with formal leaders. Journal of Personality and Social Psychology, 4, 642-655.

55.

Uttl

White

C. A.

Gonzalez

D. W.

(2016). Meta-analysis of faculty’s teaching effectiveness: Student evaluation of teaching ratings and student learning are not related. Studies in Educational Evaluation, 54, 22-42. doi:10.1016/j.stueduc.2016.08.007

56.

Van den Bos

. (2001). Uncertainty management: the influence of uncertainty salience on reactions to perceived procedural fairness. Journal of personality and social psychology, 80(6), 931.

57.

Van den Bos

. (2005). What is responsible for the fair process effect? In Greenberg

Colquitt

J. A.

(Eds.), Handbook of organizational justice (pp. 273-300). Mahwah, NJ: Lawrence Erlbaum.

58.

Wendorf

C. A.

Alexander

(2005). The influence of individual- and class-level fairness-related perceptions on student satisfaction. Contemporary Educational Psychology, 30, 190-206.