Quality of Work and Team Spirit as Drivers of Student Peer Evaluation on Advertising Group Project Performance

Abstract

Using the evaluation traits typically adopted by college instructors, this study investigates the dimensions of peer evaluation criteria used by college students in advertising management and campaigns courses and examines their effects on overall peer evaluation under the influence of three moderating self-related concepts: self-esteem, self-competence and goal orientations (i.e., learning goal orientation and performance goal orientation). The results, from Study 1 suggest Quality of Work and Team Spirit as two important factors for students’ evaluation of their peers. The moderating effects of self-related concepts from Study 2 indicate that students with high self-esteem and high self-competence use Quality of Work predominantly, while students with high learning goal orientation and performance goal orientation use both Quality of Work and Team Spirit for peer evaluations. Overall, the findings indicate that students use Quality of Work more strongly in evaluating their team peers, compared to Team Spirit.

Using teams in business to actively adapt to the fast-changing business environment has become a prevalent and typical trend (May & Gueldenzoph, 2006; Ghorpade, 1993; Hackman, 2002; Katzenbach & Smith, 1993). The increased weight placed on team or group work has made collaborative skills some of the best virtues employees can have. A quick glance at the job announcements in classified ads confirms that companies are seeking employees who are effective team players (Gueldenzoph & May, 2002). For example, Gueldenzoph and May (2002) mentioned that Ford Motor Company was emphasizing collaborative skills, but Ford still considers it as an important driver of the company's success: “Working as One… We've grown as a market leader thanks to our skilled and motivated team” (http://corporate.ford.com/careers). Reflecting this, collaborative team learning has often been adopted in both secondary and post-secondary classrooms (Gueldenzoph & May, 2002).

The growing trend of using teams or work groups has necessitated a critical change in employee performance assessment processes because managers or supervisors often cannot be in a position to carry out the conventional supervisor-employee performance evaluation. When evaluating each member of a team, peers may be in the best position to provide relevant assessment of individuals (May & Gueldenzoph, 2006). Peer evaluation, defined as an assessment process where team members evaluate each other on a given set of performance or personal characteristics against a set of rating scales (Kane & Lawler, 1978), has therefore been regarded as one of the effective remedies used for performance evaluation of individuals in teams. As such, peer evaluation has also been adopted by educators of secondary and post-secondary classrooms to effectively appraise the performance of individual team members.

Educators and scholars have developed peer evaluation instruments with acceptable levels of reliability and validity (e.g., DeNisi & Mitchell, 1978). In addition, studies have focused on some important issues pertaining to the accuracy of peer evaluations by examining the problems of rater or measurement errors (e.g., Murphy & Cleveland, 1991). Another body of studies has investigated the factors influencing raters and the process of peer evaluations. However, while there are many valuable studies focusing on various topics of peer evaluation, there are yet unknown insights as to (1) the evaluation criteria the raters or students use when evaluating other team members and (2) the raters’ personal factors that may influence or interfere with the peer evaluation process.

The purpose of this study, therefore, is to (1) empirically verify and find the dimensions of peer evaluation criteria used by raters, (2) identify the personal contextual factors that can either directly or indirectly influence the process of peer evaluation and (3) provide some critical consideration factors that need to be incorporated in order to get more accurate and unbiased insights from peer evaluation results.

Literature Review Benefits and Issues of Peer Evaluation

In education, Pond and Ul-Haq (1997) define peer evaluation as “an assessment methodology that allows students to provide input into the assessment procedure through evaluating each others’ performance in out-of-class learning activities, with control of the final grade remaining with the teacher” (p. 331). Literature shows that peer evaluations can provide two functions in the performance appraisal of individual team members: a formative function and a summative function (Chen & Lou, 2004). The formative function can help both individual team members and the team. Peer evaluation can help individuals modify their in-group behaviors as necessary to ensure their grade is representative of their effort. This individual-level motivation can ultimately facilitate the overall group performance and dynamics (Gueldenzoph & May, 2002). The summative function of peer evaluation provides the supervisors or instructors with useful information and a high level of feedback specificity for conducting performance appraisals of individual team members. The composite and summative rating scores provided by other group members offer a more accurate assessment of performance (Sherrard & Raafat, 1994).

In spite of its usefulness, peer evaluation has several potential problems that can limit its effectiveness. Most problems are due to the measurement errors that can result from the way the evaluation is designed, the testing environment and factors related to the individual raters (Kennedy, 2005). Some raters may give relatively accurate ratings of their peers. However fatigue due to the requirement that they complete a long evaluation questionnaire for each peer in their group can affect their ability to concentrate. Occasionally, rating scores can be an over- or under-estimate of true rating because raters sometimes report scores strategically hoping to make their own scores stand out as relatively higher. In addition, raters can also evaluate the performance related areas based on non-performance related areas such as friendliness, favorability or popularity (Liden & Mitchell, 1983).

The psychometric view holds that peer evaluations are not usually accurate due to the existence of rater errors (Imada, 1982; Murphy & Cleveland, 1991) and also due to the fact that peer evaluations usually employ behaviorally-anchored measures that can misdirect attention and retrieval processes (Murphy, Martin & Garcia, 1982), especially when compared to other methods such as peer rankings or peer nominations (Kane & Lawler, 1978). Rater errors can also be caused by several factors such as a rater's inexperience with the peer assessment procedure, which often results in undifferentiated marks (Kennedy, 2005), labor intensiveness of evaluation (especially when there are many team members), a rater's lack of motivation or willingness to provide accurate ratings for the purpose of maintaining the social integrity of the group (i.e., team members may hesitate to differentiate the peer performances when such distinctions threaten a positive team climate) (Liden & Mitchell, 1983) and the performance level of a rater (i.e., better performers generally provide more accurate ratings) (Landy & Farr, 1983). Instructors’ and supervisors’ reluctance to use peer evaluation is also problematic due to their own disbelief about the validity of the process. They may also avoid peer evaluation because of students’ concerns regarding the assessment bias based on nonperformance factors such as arbitrary and irrelevant grading standards peer raters may use.

Though no clear solution has yet been found, several studies provide useful recommendations to fix some problems. Extant literature provides two general approaches to address the problems: improving (1) rating instruments and (2) raters’ rating skills. Miller (2003) studied the effect of rating criteria specificity on peer evaluation accuracy and suggested that a highly specific assessment instrument, rather than a more global instrument, produces better quantitative differentiations of performance level. But studies suggest that the rating scale modification alone may not be sufficient to improve the performance rating (Gomez-Mejia, 1988). In addition to the scale modifications, researchers have suggested rater training as a complementary way to improve the accuracy of peer evaluations. For example, Smith (1986) identified three training methods for improved accuracy of peer ratings: (1) rater error training, (2) performance dimension training and (3) performance standards training. Woehr and Huffcutt (1994) identified an additional training methodology, behavioral observation training (for a detailed review, see May and Gueldenzoph, 2006).

Even if rater training is one of the options instructors can use to enhance the accuracy of peer evaluation, it is difficult to train students effectively because of the limited time instructors can use outside of normal classroom hours and the limited interest students might have in peer evaluation training. By knowing a simplified set of criteria students use when they evaluate their peers on the traditional peer evaluation scale items, instructors may be able to effectively and efficiently “train” their students more formatively.

Criteria for Peer Evaluation

Peer rating instruments are commonly composed of several items that can be wrapped into two general dimensions: learning outcomes (e.g., Quality of Work) and learning process (e.g., level of participation in a team project) (Miller, 2003; Heylings & Stefani, 1997; Lopez-Real & Chan, 1999; MacAlpine, 1999). Gueldenzoph and May's (2002) extensive review of peer evaluation studies identified five criteria typically used for peer evaluations:

(1) commitment to the group, (2) ability to deal constructively with conflicts, (3) active participation in the decision-making process, (4) accountability for assigned tasks and (5) assumption of initiative or leadership role. Of these five, item four is related to the learning outcomes and the rest of the criteria are learning process related.

Though literature shows that peer rating instruments have various items ranging from specific to general, it is yet uncertain how raters or students use those items in evaluating peers. The speculations from the literature are that students may evaluate their peers in terms of the learning outcome and learning process as Miller (2003) noted, or in terms of five criteria suggested by Gueldenzoph and May (2002). However an empirical study would be necessary to confirm or disconfirm such conceptual findings. As there is no sufficient theoretical ground to suggest a hypothesis, a research question calling for an exploration of the criteria used by peer evaluators is set as the following.

RQ1: What are the perceived peer evaluation criteria used by students?

Factors Influencing Peer Evaluation

Though many peer evaluation studies have focused on the instrumental (i.e., rating scale) or executional (i.e., rating skill) aspects of peer evaluation, a few studies have expanded the research scope and investigated some factors that may directly or indirectly influence the raters in assessing their peers. For example, Persons (1998) identified two general peer evaluation influencers: (1) ex-ante factors and (2) ex-post factors. The ex-ante factors are conditional influencers existing before peer evaluations, including gender, race, major of students, previous GPA and prior background relevant to the team task. The ex-post factors are those influencers occurring after the team project, such as exam scores, class participation and group assignment grades. Some other factors found in the literature include rater competence (Barclay & Harland, 1995), rater's self-esteem (Duffy, Shaw & Stark, 2000), culture and in-/out-group membership (Gomez, Kirkman & Shapiro, 2000), characteristics of organization (Landy & Farr, 1980), attitudes of team members toward the peer evaluation process (Fedor, Bettenhausen & Davis, 1999) and personality characteristics such as conscientiousness (Tziner, Murphy & Cleveland, 2002).

Of several external factors identified in the literature, the present study focuses on some of the self-related psychological variables that might moderate the peer rating process. The variables investigated in the current study are self-esteem, self-competence and two goal orientations (i.e., learning and performance goal orientation). By notifying the students about the potential influence of self-related psychological conditions such as self-esteem/competence or their learning goal orientations, instructors may be able to reduce the measurement errors caused by such individual factors and ensure the accuracy of peer evaluations.

Research Design Overview

Two studies were conducted. In Study 1, exploratory factor analysis was performed in order to find the evaluation criteria used by students, using data from four semesters of peer evaluation responses collected from 201 undergraduate students. Upon finding the evaluation criteria for RQ1, the first hypothesis was suggested to test the predictive validity of peer evaluation criteria found from RQ1. Other hypotheses (H2–5) pertaining to the relationships between peer evaluation criteria identified from RQ1 and each self-related psychological variable (i.e., self-esteem, self-competence and two goal orientations) were also formulated. A new data set then was collected in Study 2 to test the hypotheses (H2–5) proposed.

Study 1: Peer Evaluation Dimensions and Project Performance Method of Study 1

A total of 51 groups consisting of 201 undergraduate students at a major southeastern university enrolled in advertising management and advertising campaigns courses constituted the sample for Study 1. All groups were formed through self-selection. The most common group size was five students and the average group size was 4.75 students. Students completed a variety of group and individual tasks that would help to develop effective marketing communication strategies or create advertising campaign plans for specific brands or companies. Each member of the group played different roles within a team (e.g., working on account planning, research, media planning and creative executions). Group requirements included leading class discussions, participating in team meetings, providing weekly team reports, preparing the advertising campaign plans book designed to recommend advertising ideas and completing the peer evaluation. In particular, peer evaluation constituted 10% (advertising management course) or 40% (advertising campaigns course) of each individual's course grade. At the end of the academic semester, students were asked to evaluate the individual performances of their peers. Throughout the two-year period from 2006 to 2007, data were collected on peer evaluation in both ad management and ad campaigns classes.

Peer evaluation question items (see Table 1) used in Study 1 were a collection of items from various sources. The question items are very comparable to the ones found in the literature (see Gueldenzoph & May, 2002; Ohland & Layton, 2000). The items were measured with a 19-item scale, ranging from 1 (extremely poor) to 9 (excellent). In addition, a global evaluation measure, “overall peer evaluation,” was measured by a single question that asked participants to indicate how strongly they would recommend the peer in question to other teams in the future. This overall evaluation item was used as a dependent variable testing the predictive validity of evaluation criteria (for RQ 1) identified in Study 1.

Table 1

Dimensions of Peer Evaluation (N = 201)

Attributes of Peer Evaluation	Quality of Work	Team Spirit
Creativity	.845	.091
Active Role	.753	.536
Volunteerism	.740	.437
Generate ideas	.714	.291
Quality of Work	.709	.374
Shows initiative	.705	.535
Contributes to learning experience	.702	.498
Interest & Enthusiasm	.702	.474
Quantity of Work	.659	.375
Brings integrity to the project	.648	.594
Attends group meetings	.163	.866
Is a team player	.388	.810
Dependability	.463	.800
Devotes time to project	.607	.713
Cooperates with group	.226	.711
Follow through on assignments	.531	.710
Takes project seriously	.495	.690
Communicates with group	.442	.662
Met group deadlines	.547	.624
Eigenvalue	12.664	1.18
% of variance	66.651	6.21
Cumulative %	66.651	72.862

Note: N = 201; Higher factor loadings are set in bold for the clarity of presentation.

Results of Study 1

Prior to testing the hypotheses, an exploratory factor analysis (N = 201) was conducted to identify the dimensions of peer evaluation for assessing students’ performance in group projects. The factor analysis with varimax rotation generated a two-component solution explaining approximately 72.9% of the total variance. As shown in Table 1, the first factor clearly represented Quality of Work, which was characterized as creativity, volunteerism, generates ideas, quality of work and interest and enthusiasm. On the other hand, the second factor, Team Spirit, was described by peer evaluation items such as attending group meetings, being a team player, dependability, cooperating with the group and communicating with the group. The internal consistencies of each factor were satisfactory (Quality of Work's α= .96, Team Spirit's α= .97).

As the answer for RQ1 was Quality of Work and Team Spirit, it would be necessary to confirm the predictive validity of these two variables in predicting the overall peer evaluation. Accordingly, the following hypothesis was proposed.

H1: High levels of Quality of Work and Team Spirit of an individual will lead to a positive overall evaluation.

H1 was answered through multiple regression analysis to examine the relationship between the dimensions of peer evaluation and group project performance. Specifically, the dependent measure – group project performance – was regressed on two constructs of peer evaluation. The results strongly supported H1 and showed that Quality of Work (β=.60, t=10.91, p < .001) and Team Spirit (β=.35, t=6.37, p<.001) had significant effects on overall peer evaluation (R² = .85, adjusted R² = .84, F (2, 198) = 539.9, p < .001).

In addition to the first hypothesis, the interaction effect of two factors was examined to see if the effect of Quality of Work on overall peer evaluation is different for individuals with high and low levels of Team Spirit. The result indicated that the interaction effects was not statistically significant (β = –.003, p > .05), indicating that the effect of Quality of Work on group project performance does not differ according to different levels of Team Spirit.

Study 2: Moderating Roles of Self-Related Variables Overview of Study 2

Study 1 found two dimensions of peer evaluation for group projects and confirmed that high levels of Quality of Work and Team Spirit can predict the overall peer evaluation. The goal of Study 2 was to establish the relationship among peer evaluation dimensions and self-related psychological variables (i.e., self-esteem, self-competence and two goal orientations). In Study 2, first the relevant literature pertaining to the four self-related psychological variables considered to be related to the peer evaluation process was reviewed. The hypotheses were generated based on the literature findings.

Motivation of Self and Self-Related Concepts

Literature suggests that the self is conceived as an actively stabilized cognitive system that defends itself from external/internal forces or information that intimidates its consistency, stability and continuity (Greve & Wentura, 2003; Linville & Carlston, 1994). There are many self-related concepts introduced in the literature, but the current study focused on two frequently used concepts, self-esteem and self-competence, and two goal orientations that are related to educational context (e.g., peer evaluation).

Self-Esteem

Self-esteem refers to an overall attitude (positive or negative) toward the self (Tafarodi & Swann, 1995). Literature shows that changes in self-evaluations such as self-esteem can lead to changes in evaluations of self and others (Shrauger & Terbovic, 1976). Different levels of self-esteem are known to influence people's self evaluations. High-self-esteem individuals are described as defensive (Cohen, 1959), thus they try to avoid the recall of negative aspects in themselves (Silverman, 1964) and try to recall their performance as more positive than it really was (Crary, 1966). On the other hand, low-self-esteem individuals are known to recall more failure-related performances than success-related performances (Silverman, 1964). As self-esteem is viewed as influencing one's evaluation of self and others, it would be important to examine how self-esteem affects a rater's rating procedure and strategy.

In peer evaluation, students with high self-esteem try to evaluate themselves more positively than their performance actually was. In addition, their high self-esteem may influence their evaluation of other team members. As they try to maintain their self-concept (i.e., high self-esteem), they may rate high-performing team members more favorably compared to low-performing members. Therefore, the main evaluation criteria for individuals with high self-esteem would be Quality of Work compared to Team Spirit. This suggests the following hypothesis.

H2: For the individuals with high self-esteem, Quality of Work will have a much stronger effect on evaluating peers than will Team Spirit.

Self-Competence

Self-competence is the overall sense of oneself as effective, capable and confident (Tafarodi & Swann, 1995). Though self-competence has long been considered as a part of self-esteem, they are different because self-competence is more related to one's capability (either perceived or real) while self-esteem is related to more affective and global attitudes toward the self. Tafarodi and Swann (1995) explain that self-competence results from the successful management of one's task environment. It is internally regulated in that we know what we want (intentions) through our own efforts and what we get (outcomes). If there is a correspondence between intentions and outcomes, we sense this as an achievement due to our own efforts, and then our self-competence is increased. Therefore an individual's cumulative experience of success increases “an affectively charged sense of self as a locus of potential power” (Tafarodi & Swann, 1995, p. 325).

A student with high self-competence would have a high intention that is linked to a high outcome. Since the group project cannot be done alone, high-self-competence students will try to favorably evaluate team members who meet their quality standards, which can help achieve the high outcome that brings in the intention-outcome correspondence experience. The following hypothesis is suggested for the effect of self-competence on the peer evaluation process.

H3: For the individuals with high self-competence, Quality of Work will have a much stronger effect on evaluating peers than will Team Spirit.

Goal Orientations

Goal orientation is conceived as a construct that shows the purpose-related state of mind when an individual faces a task. Individuals normally have one of two goal orientations toward tasks: either a learning or performance orientation (Dweck, 1986). A learning orientation reflects a desire to increase one's task competence, while a performance orientation is characterized by a desire to perform well and to be positively rated by others (Farr, Hofmann & Ringenbach, 1993). Individuals possessing a high learning goal orientation may approach tasks in order to develop their abilities and skills. On the other hand, individuals with a high performance goal orientation approach tasks with the intention of performing well. Therefore, in peer evaluation, students with high learning-oriented goals would approach team tasks with the intention of improving and possibly mastering skills. This may make the high learning goal-oriented students appraise team members in terms of both Quality of Work and Team Spirit. On the other hand, students with high performance-oriented goals may appraise team members based more on excellence in performance (i.e., Quality of Work). The following hypotheses are formulated based on these speculations.

H4: For the individuals with high learning-oriented goals, both Quality of Work and Team Spirit will have a significant effect on evaluating peers.

H5: For the individuals with high performance-oriented goals, Quality of Work will have a much stronger effect on evaluating peers as compared to Team Spirit.

Methods of Study 2

The sample for Study 2 consisted of 28 groups, including 106 undergraduate students at a major southeastern university enrolled in advertising management and advertising campaigns courses. Note that this student sample was independent of the sample used in Study 1. In addition to measuring the peer evaluation scale, the procedures used in Study 2 were identical to those in Study 1 with the exception of self-related psychological measures. All items used in this study were measured using seven-point Likert scales (1 = strongly disagree and 7 = strongly agree) (See Table 2).

Table 2

Self Related Measures

Constructs	Scale Items
Learning goal orientation (α = .85)	1. The opportunity to do challenging work is important to me. 2. When I fail to complete a difficult task, I plan to try harder the next time I work on it. 3. I prefer to work on tasks that force me to learn new things. 4. The opportunity to learn new things is important to me. 5. I do my best when I'm working on a fairly difficult task. 6. I try hard to improve on my past performance. 7. The opportunity to extend the range of my abilities is important to me. 8. When I have difficulty solving a problem, I enjoy trying different approaches to see which one will work.
Performance goal orientation (α = .84)	1. I prefer to do things that I can do well rather than things that I do poorly. 2. I'm happiest at work when I perform tasks on which I know that I won't make any errors. 3. The things I enjoy the most are the things I do the best. 4. The opinions others have about how well I can do certain things are important to me. 5. I feel smart when I do something without making any mistakes. 6. I like to be fairly confident that I can successfully perform a task before I attempt it. 7. I like to work on tasks that I have done well on in the past. 8. I feel smart when I can do something better than most other people.
Self-esteem (α = .85)	1. I feel that I'm a person of worth, at least on an equal plane with others. 2. I feel that I have a number of good qualities. 3. All in all, I am inclined to feel that I am failure. (R) 4. I am able to do things as well as most other people. 5. I feel I do not have much to be proud of. (R) 6. I take a positive attitude toward myself. 7. On the whole, I am satisfied with myself. 8. I wish I could have more respect for myself. (R) 9. I certainly feel useless at times. (R) 10. At times I think I am no good at all. (R)
Self-competence (α = .87)	1. Owing to my capabilities, I have much potential. 2. I don't succeed at much. (R) 3. I have done well in life so far. 4. I perform very well at a number of things. 5. I am a capable person. 6. I do not have much to be proud of. (R) 7. I am talented. 8. I am not very competent. (R) 9. I deal poorly with challenges. (R) 10. I perform inadequately in many important situations. (R)

Note: All items are measured on seven-point Likert scales where 1 = strongly disagree and 7 = strongly agree; (R) indicates a reverse-coded item.

Results of Study 2

Hypotheses 2–5 predicted that the relationships between the dimensions of peer evaluation and the overall peer evaluation would be moderated by self-related factors. Based on a median split, different levels of self-related factors were divided into high and low groups. By using dummy coding for self-related factors (e.g., 1 = high levels of self-esteem and 0 = low levels of self-esteem), a series of multiple regression analyses were performed to investigate how the different levels of self-related concepts would moderate the relationship between the two peer evaluation dimensions (i.e., Quality of Work and Team Spirit) and overall peer evaluation (See Table 3).

Table 3

Results of Multiple Regression

Self-Related Concepts	Standardized coefficients (β) of independent variables
Self-Esteem	Quality of Work	Team Spirit
Low group	.648^***	.375^***
High group	.825^***	.131
Self-Competence	Quality of Work	Team Spirit
Low group	.621^***	.448^***
High group	.775^***	.175
Learning Goal	Quality of Work	Team Spirit
Low group	.640^***	.329^***
High group	.748^***	.260^***
Performance Goal	Quality of Work	Team Spirit
Low group	.661^***	.312^***
High group	.714^***	.267^**

* p < .05

p < .01

***

p < .001

For H2, the findings indicated that the relationship between Quality of Work and overall peer evaluation was significant and positive (β = .83, p < .001) when the level of self-esteem was high, whereas the relationship between Team Spirit and overall peer evaluation was not statistically significant (β = .13, p>.05) when the level of self-esteem was low. Therefore, the results confirmed that Quality of Work plays a stronger role in affecting overall peer evaluation than Team Spirit for the individuals with high levels of self-esteem.

The results of H3 showed that Quality of Work's influence on overall peer evaluation was significant (β= .76, p< .001) but the effect of Team Spirit was not significant (β= .18, p>.05), when the level of self-competence was high. This implies that Quality of Work may have a much stronger effect on overall peer evaluation than Team Spirit in the individuals with high self-competence. Therefore, H3 is supported.

The results related to H4 revealed that both Quality of Work (β = .75, p < .001) and Team Spirit (β = .26, p < .001) had significant effects on overall peer evaluation when the learning goal was high. Although the beta coefficient of Quality of Work was much higher than that of Team Spirit, H4's prediction was about the significance of both variables regardless of the effect sizes, thus the results are considered to support H4.

The results of H5 showed that the individuals with high performance-oriented goals had stronger relationships between Quality of Work and overall peer evaluation (β = .71, p < .001) compared to the relationship between Team Spirit and overall peer evaluation (β = .27, p < .001). Therefore, H5 was supported.

Discussion

The purpose of this study was to empirically identify the dimensions of peer evaluation criteria used by students and find the self-related variables that can influence the peer evaluation process. The results provide critical consideration factors that may need to be incorporated in order to get more accurate insights from the peer evaluations reported by students.

The two dimensions identified from RQ1, Quality of Work and Team Spirit, showed their predictive validity through H1. This finding empirically supports Miller's (2003) conceptualization of peer evaluation criteria: learning outcomes (e.g., quality of presentation, assignment or other product) and learning process (e.g., level of participation in a team project). It is adequate and acceptable to find that students also use similar criteria to judge their peers as their instructors do (i.e., Quality of Work would be more related to the learning outcomes and Team Spirit might be more relevant to the learning processes).

The first hypothesis (H1) testing the significant role of both Quality of Work and Team Spirit confirmed that a student's peer evaluation score depends on these two factors. Rather than explaining about and training for numerous peer evaluation items, educators may be able to effectively emphasize these two criteria so that they are easy for students to remember. However, it would be important for instructors to help student teams define what is meant by Quality of Work and Team Spirit in order to avoid a situation in which a student might think he/she is doing quality work or showing team spirit, but be surprised to discover that others on the team don't feel that way.

Hypotheses 2–5 tested the moderating effects of students’ self-related variables on the relationship between overall peer evaluation and two evaluation dimensions. Study 2 found significant moderating roles of self-esteem, self-competence and two goal orientations in the peer evaluating process. Quality of Work had the only significant and strong effect on overall peer evaluation, compared to Team Spirit, when the individual had high self-esteem. Results of H3 indicated that Quality of Work had the only significant and strong effect on peer evaluation compared to Team Spirit when the rater had high self-competence. For low self-esteem and low self-competence individuals, however, both Quality of Work and Team Spirit were significant, though Quality of Work still showed stronger impact on peer evaluation. In addition, Quality of Work showed a much stronger effect on peer evaluation compared to Team Spirit when the student had a high level of learning-oriented goals (H4) and high performance-oriented goals (H5). The results of H4 and H5 show that students at most times care more about the Quality of Work than Team Spirit regardless of type and level of goal orientations.

Overall, the results suggest that instructors of student teams may need to teach students how their self-related concepts could influence their evaluation of peers. Emphasizing and warning students about the likelihood of self-concept interference in peer evaluation would enhance the validity of peer evaluation because the students may be more careful and try harder to be unbiased when evaluating their peers.

The results of this study demonstrate that two criteria (Quality of Work and Team Spirit) are important dimensions and the effects of those criteria on peer evaluation can be moderated by some psychological self-concepts identified in this study. Future studies may incorporate other self concepts such as self-worth and self-liking. In addition, employing other extraneous variables such as gender, race, major, previous GPA and prior background pertinent to the team task would provide more detailed and useful findings. Comparing and investigating the effect of the instructor emphasizing the possibility of self-concept interference in peer evaluation among students would also offer practical insights.

References

Barclay

J. H.

, & Harland

L. K.

(1995). Peer performance appraisals: The impact of rater competence, rater location, and rating correctability on fairness perceptions.. Group & Organization Management, 20(1), 39–60.

Chen

, & Lou

(2004). Students’ perceptions of peer evaluation: An expectancy perspective. Journal of Education for Business, 79(5), 275–282.

Cohen

A. R.

(1959). Some implications of self-esteem for social influence. In Hovland

C.I.

& I.

Janis

(Eds.), Personality and Persuasibility (pp. 102–120). New Haven, CT: Yale University Press.

Crary

W. G.

(1966). Reactions to incongruent self-experience. Journal of Consulting Psychology, 30(3), 246–252.

DeNisi

A. S.

, & Mitchell

J. L.

(1978). An analysis of peer ratings as predictors and criterion measures and a proposed new application. Academy of Management Review, 3, 369–374.

Duffy

M. K.

, Shaw

J. D.

, & Stark

E. M.

(2000). Performance and satisfaction in conflicted interdependent groups: When and how does self-esteem make a difference? Academy of Management Journal, 43(4), 772–782.

Dweck

C. S.

(1986). Motivational processes affecting learning. American Psychologist, 41(10), 1040–1048.

Farr

J. L.

, Hofmann

D. A.

, & Ringenbach

K. L.

(1993). Goal orientation and action control theory: Implications for industrial and organizational psychology. In. In Cooper

C. L.

& Robertson

I. T.

(Ed.), International review of industrial and organizational psychology (pp. 193–232). New York: Wiley.

10.

Fedor

D. B.

, Bettenhausen

K. L.

, & Davis

(1999). Peer views: Employee's dual roles as raters and recipients. Group & Organization Management, 24(1), 92–120.

11.

Ghorpade

(1993). Managing five paradoxes of 360-degree feedback. Academy of Management Executive, 14(1), 140–150.

12.

Gomez-Mejia

L. R.

(1988). Evaluating employee performance: Does the appraisal instrument make a difference? Journal of Organizational Behavior Management, 9, 155–172.

13.

Gomez

, Kirkman

B. L.

, & Shapiro

D. L.

(2000). The impact of collectivism and in-group/out-group membership on the evaluation generosity of team members.. Academy of Management Journal, 43(6), 1097–1106.

14.

Greve

, & Wentura

(2003). Immunizing the self: Self-concept stabilization through reality-adaptive self-definitions. Personality and Social Psychology Bulletin, 29(1), 39–50.

15.

Gueldenzoph

L. E.

, & May

G. L

(2002). Collaborative peer evaluation: Best practices for group member assessments. Business Communication Quarterly, 65(1), 9–20.

16.

Hackman

J. R.

(2002). Leading teams: Setting the stage for great performances. Boston, MA: Harvard Business School Press.

17.

Heylings

D. J.

, & Stefani

L. A.

(1997). Peer assessment feedback marking in a large medical anatomy class. Medical Education, 31(4), 281–286.

18.

Imada

A. S.

(1982). Social interaction, observation, and stereotype as determinants of differentiation in peer rating. Organizational Behavior and Human Performance, 29, 397–415.

19.

Kane

J. S.

, & Lawler

E. E.

(1978). Methods of peer assessment. Psychological Bulletin, 85(3), 555–586.

20.

Katzenbach

J. R.

, & Smith

D. K.

(1993). The discipline of teams. Harvard Business Review,111-120.

21.

Kennedy

G. J.

(2005). Peer-assessment in group projects: Is it worth it? Proceedings of the Australasian Conference on Computing Education, 42, 59–65.

22.

Landy

F. J.

, & Farr

L. L.

(1980). Performance Rating. Psychological Bulletin, 87, 72–107.

23.

Landy

F. J.

, & Farr

L. L.

(1983). The measurement of work performance: Methods, theory, and applications. New York: Academic Press.

24.

Liden

R. C.

, & Mitchell

T. R.

(1983). The effects of group interdependence on supervisor performance evaluations. Personnel Psychology, 36(2), 289–299.

25.

Linville

, & Carlston

D. E.

(1994). Social cognition perspective on self. In Devine

P. G.

Hamilton

D. L.

, & Ostrom

T. M.

(Eds.), Social cognition: Contributions to classic issues in social psychology (pp. 143–193). New York: Academic Press.

26.

Lopez-Real

, & Chan, Y-P. R. (1999). Peer assessment of a group project in a primary mathematics education course. Assessment & Evaluation in Higher Education, 24(1), 67–79.

27.

MacAlpine

J. M. K.

(1999). Improving and encouraging peer assessment of student presentations. Assessment & Evaluation in Higher Education. 24(1), 15-25.

28.

May

G. L.

, & Gueldenzoph

L. E.

(2006). The effect of social style on peer evaluation ratings in project teams. Journal of Business Communication, 43(1), 4–20.

29.

Miller

P. J.

(2003). The effect of scoring criteria specificity on peer and self-assessment. Assessment & Evaluation in Higher Education, 28(4), 383–394.

30.

Murphy

K. R.

, & Cleveland

J. N.

(1991). Performance appraisal: An organizational perspective. Boston, MA: Allyn and Bacon.

31.

Murphy

K. R.

, Martin

, & Garcia

(1982). Do behavioral observation scales measure observation? Journal of Applied Psychology, 67, 562–567.

32.

Ohland

M.W.

, & Layton

R. A.

(2000). Comparing the reliability of two peer evaluation instruments. Paper presented at the annual conference of the American Society for Engineering Education. St. Louis, MO.

33.

Persons

O. S.

(1998). Factors influencing students’ peer evaluation in cooperative learning. Journal of Education for Business, 73(4), 225–229.

34.

Pond

, & Ul-Haq, R. (1997). Learning to assess students using peer review. Studies in Educational Evaluation, 23(4), 331–348.

35.

Sherrard

W. R.

, & Raafat

(1994). An empirical study of peer bias in evaluations: Students rating students. Journal of Education for Business, 70(1), 43–47.

36.

Shrauger

J. S.

, & Terbovic

M. L.

(1976). Self-evaluation and assessments of performance by self and others. Journal of Consulting and Clinical Psychology, 44(4), 564–572.

37.

Silverman

(1964). Differential effects of ego threat upon persuasibility for high and low self-esteem subjects. Journal of Abnormal and Social Psychology, 69, 567–572.

38.

Smith

D. E.

(1986). Training programs for performance appraisal: A review. Academy of Management Review, 11(1), 22–40.

39.

R. W., & Swann

W. B.

(1995). Self-liking and self-competence as dimensions of global self-esteem: initial validation of a measure.. Journal of Personality Assessment, 65(2), 322–342.

40.

Tziner

, Murphy

, & Cleveland

(2002). Does conscientiousness moderate the relationship between attitudes and beliefs regarding performance appraisal and rating behavior? International Journal of Selection and Assessment, 10, 218–224.

41.

Woehr

D. J.

, & Huffcutt

A. I.

(1994). Rater training for performance appraisal: A quantitative review. Journal of Occupational & Organizational Psychology, 67(3), 189–205.