Factors Influencing Achievement in Undergraduate Social Science Research Methods Courses

Abstract

Undergraduate social science research methods courses tend to have higher than average rates of failure and withdrawal. Lack of success in these courses impedes students’ progression through their degree programs and negatively impacts institutional retention and graduation rates. Grounded in adult learning theory, this mixed methods study examines the factors that influence student achievement in these courses among a sample of 724 social science students. Quantitative results indicate math self-concept, the belief that being good at math is necessary for success in the course; anxiety; attributions of course utility; learning approach; and GPA predict perceived learning. Qualitative results suggest students’ research self-concepts shape whether they take a deep learning approach (leaning in) or a surface learning approach (resistance) to the course. Course instructors also impact students’ perceptions of learning.

Keywords

research methods scholarship of teaching and learning student learning

Undergraduate social science research methods courses tend to have higher than average rates of failure and withdrawal (Macheski et al. 2008). As courses required for the major, they are mandatory and usually require a minimum C grade to receive credit. Consequently, many students repeat these courses, and some are forced out of the major. Lack of success in these courses impedes student progression and negatively impacts institutional retention and graduation rates.

Research indicates social science research methods courses are problematic for students and instructors alike (Blalock 1987; Earley 2014). Student attitudes toward these courses tend to be negative and sometimes even hostile (Bos and Schneider 2009; Murtonen and Lehtinen 2003; Parker et al. 2008; Rancer, Durbin, and Lin 2013). Much of the literature on teaching research methods courses focuses on class exercises or strategies designed to teach specific skills (Leston-Bandeira 2013; Stalp and Grant 2001; Tan and Ko 2004; Taylor and McConnell 2001). Only a few studies have examined achievement in social science research methods courses (Crowe, Silva, and Ceresola 2015; Greene and Miller 1996; Núñez-Peña, Súarez-Pellicioni, and Bono 2013; Rancer et al. 2013; Sizemore and Lewandowski 2009).

This mixed methods study examines the factors that influence achievement in undergraduate social science research methods courses. A mixed methods approach enables an expanded analysis of the issues associated with achievement, increasing our understanding of students’ struggles to succeed. Such an understanding is important to faculty who teach these courses and to others interested in improving both course and degree completion rates.

Theoretical Framework and Literature Review

Illeris’s (2003, 2009) theory of adult learning provides a useful framework for examining student achievement in undergraduate research methods courses. According to this theory, learning entails two distinct processes: an internal process of acquisition and an external process of interaction. These learning processes occur along three dimensions: the psychodynamic dimension of attitudes, beliefs, and feelings; the cognitive dimension of knowledge and skills; and the environmental dimension encompassing the learner’s social, cultural, and material contexts. The internal process of acquisition occurs within the psychodynamic and cognitive dimensions, while the external process of interaction occurs within the environmental dimension. In addition to these processes and dimensions, “conditions of learning” influence learning indirectly (Illeris 2009). Internal conditions of learning are characteristics of the learner such as academic ability and age. External conditions of learning include characteristics of the learning situation such as school or workplace. In order to better understand student achievement in social science research methods courses, it is necessary to consider multiple aspects of learning.

Psychodynamic Factors

Prior to taking the course, most students have a poor understanding of the nature of research (Bos and Schneider 2009; Earley 2014). They fail to see how research connects to their discipline of study (Leston-Bandeira 2013) and may feel forced into taking the course (Macheski et al. 2008). Students assume that proficiency in math is necessary for success in the course, although most topics covered in the course are not directly math related (Murtonen and Lehtinen 2003; Parks, Faw, and Goldsmith 2011).

Among students in a psychology research methods course, perceived usefulness of the course was positively associated with achievement as measured by knowledge of course topics (Sizemore and Lewandowski 2009). Although students’ knowledge of research methods increased over the course, their perceptions of its utility decreased. In general, students fail to appreciate the importance of the skills learned in the course. A majority of students deem the course irrelevant and unnecessary for their degree or their future careers (Earley 2014; Macheski et al. 2008).

Math anxiety was negatively associated with course grade among psychology students in a research design course while self-confidence in math ability was positively associated with course grade (Núñez-Peña et al. 2013). Math anxiety was positively related to perceived difficulty of course topics and negatively related to understanding of course topics in a communications research methods course (Rancer et al. 2013).

Math anxiety consists of “feelings of tension and anxiety that interfere with the manipulation of numbers and the solving of mathematical problems in a wide variety of ordinary life and academic situations” (Richardson and Suinn 1972:551). It is a learned emotional response (Tobias 1993), likely a consequence of the widely held belief that individuals are either “math people” or “not math people.” A significant proportion of social science students approach research methods courses with high levels of math anxiety (Bos and Schneider 2009; Macheski et al. 2008), statistics anxiety (Blalock 1987; Onwuegbuzie and Wilson 2003), and general anxiety about the course (Earley 2014).

According to the implicit theories model (Dweck, Chiu, and Hong 1995), certain personal attributes, such as intelligence, are considered by many to be either fixed or changeable. This concept of trait beliefs has been extended to math as well, with important implications: Women who believed their math abilities were fixed were less interested in math and less likely to pursue a career in math compared to women who believed their math abilities were changeable (Burkley et al. 2010). Math self-concept, a dimension of academic self-concept, represents an individual’s evaluation of his or her math ability (Marsh 1993). Research indicates a strong association between academic domain self-concept (e.g., math self-concept) and achievement in that domain (Marsh and Seaton 2013). In a study of sociology students, a majority described themselves as “non-mathematical persons,” concluding that they could not learn research methods (Murtonen and Lehtinen 2003).

Cognitive Factors

Students experience research methods courses as more difficult than other content-specific courses (Macheski et al. 2008). They have trouble understanding abstract concepts and research terminology (Murtonen and Lehtinen 2003). In a study examining the effects of peer review on student performance in a sociological research methods course, researchers expected that the peer review process would promote critical thinking and higher-order levels of learning but found it did not improve assignment grades or final course grades (Crowe et al. 2015).

Students choose from two contrasting approaches to learning based on intent (Biggs 1988). Those who intend to just meet course requirements adopt a “surface learning approach,” a reproductive strategy focused on memorization, while students who intend to achieve competence in the course subject adopt a “deep learning approach,” a meaningful strategy focused on understanding and integration of previous knowledge. Learning approach has been associated with academic achievement among students in psychological research methods courses (Diseth and Martinsen 2003; Greene and Miller 1996). A surface learning approach was associated with lower exam grades.

Environmental Factors

Undergraduate instructors have a significant impact on the learning environment through their pedagogical choices and establishment of classroom climate (Hirschy and Wilson 2002). Instructor/student interactions and student perceptions of instructor inclusiveness and fairness influence student effort. Analysis of national data indicates faculty attitudes and behaviors play an important role in creating a culture conducive to student engagement and learning (Umbach and Wawrzynski 2005). Active learning assignments and cross-discipline “guest discussion facilitators” are associated with increased student engagement and interest in research (Pfeffer and Rogalin 2012). Sociology students have attributed their difficulties in research methods courses to “bad teaching” and course structure (Murtonen and Lehtinen 2003). They described research methods courses as taught at superficial levels with too much material to be covered adequately and proficiency with analytic software emphasized over understanding of methodological procedures.

The purpose of this study was to identify psychodynamic, cognitive, and environmental factors that influence achievement in undergraduate social science research methods courses and develop a better understanding of how students experience these courses.

Method

This study used an embedded mixed methods design (Creswell and Plano Clark 2011), in which both quantitative and qualitative data were collected at the same time through an online survey. It was conducted at a public university in the Southeastern United States with an undergraduate enrollment of 22,000. The study received Institutional Review Board approval. The university’s Office of Institutional Research provided a list of all currently enrolled students who had taken undergraduate research methods courses in communications, criminal justice, political science, psychology, and sociology within the past three years. These disciplines were chosen because the content of the research methods courses was similar. All 2,498 students on the list were sent an email inviting them to take the online survey. The university’s Institute of Public Research deployed the survey. Completed surveys were sent to the Office of Institutional Research where all identifying information was removed and GPA, age, race/ethnicity, gender, and final course grade were added; then the data set was sent to the author.

Participants

A total of 724 surveys were completed, for a response rate of 29 percent. The majority of participants were women (74 percent). Almost two-thirds (63 percent) identified as white, 18 percent as African American, 8 percent as Hispanic, and 10 percent as Asian American and other. The mean age of participants was 22.6. Just about one-fourth were sociology majors; the rest were psychology (23 percent), communication (20 percent), political science (17 percent), and criminal justice (16 percent) majors. Successful students were overrepresented; only 14 percent of participants failed to pass the class with a C or above, compared to 23 percent of students in the target population. See Table 1 for a comparison of course grade distribution between participants and all students taking these courses over the three-year period. The sample reflected the distribution of majors and racial/ethnic makeup of the target population, although women were somewhat overrepresented (64 percent of the target population were women). Participants’ mean GPA was 2.98. To adjust for response bias, a weighted sample was constructed on the basis of final course grade. Parallel sets of analyses were conducted with both weighted and unweighted samples and yielded analogous results. The analyses reported in this article are from the weighted sample.

Table 1.

Sample (n = 724) and Target Population Percentage Grade Distribution.

Discipline	Percentage Grade Distribution
	Sample				Target Population
	A	B	C	D/F	A	B	C	D/F
Sociology (n = 178)	41.1	29.2	16.8	12.9	29.3	28.1	16.8	25.8
Psychology (n = 165)	22.6	33.9	26.0	17.5	13.8	29.9	24.5	31.8
Communication (n = 144)	34.1	43.0	16.7	6.2	29.1	43.7	18.9	8.3
Political science (n = 122)	30.4	36.8	16.4	16.4	17.4	31.6	24.3	26.7
Criminal justice (n = 115)	30.5	31.3	23.4	14.8	23.0	33.7	22.8	20.6
Total 724	31.9	34.7	19.9	13.5	21.2	34.2	22.0	22.6

Measures

Two measures of achievement were used. One was final course grade. The second, perceived learning, was derived by summing responses to five items from Sizemore and Lewandowski (2009) (e.g., “I remember most of the things I learned in my research methods class”; α = .77).

Age reflects students’ age at the time the survey was taken. GPA is used as a proxy for academic ability as per Grove, Wasserman, and Grodner (2006) and reflects cumulative GPA for the semester prior to the semester the course was taken. Participants were asked to respond to the questions regarding math trait, math self-concept, math ability, and utility based on their perceptions prior to taking the course. Math trait was derived by summing two items from Burkley et al. (2010) (e.g., “You have a certain amount of math ability, and you can’t really do much to change it”; α = .79). Math self-concept was derived by summing responses to two items (e.g., “I am good at math”; α = .89). Math ability necessary consisted of participants’ level of agreement from (1) strongly disagree to (5) strongly agree with the statement: “To understand research methods you have to be good at math.” Utility of the course was derived by summing responses to five items from Sizemore and Lewandowski (2009) (e.g., “Research will be useful for my career”; α = .90). Participants assessed their level of anxiety on the first day of the course as low, moderate, high, or very high.

Learning approach was adapted from the Revised Two-Factor Study Process Questionnaire (Biggs, Kember, and Leung 2001). Deep learning approach was derived by summing four items (e.g., “I test myself on important topics until I really understand them”; α = .77). Surface learning approach was derived by summing responses to four items (e.g., “I find I can get by on most tests by memorizing key sections rather than trying to understand them”; α = .76). Student effort was derived by summing responses to five items (e.g., “How much time per week did you spend studying or preparing for this class?” This measure did not have an acceptable level of internal consistency (α = .57) and did not improve the fit of the regression models; however, data from the individual variables were useful for other analytic purposes.

Qualitative data were obtained by responses to the following open-ended questions: “How would you describe your overall experience of the Research Methods course?” “What did you hear about the course prior to taking it?” “What, if anything, about the course made you anxious?” “What enabled you to be successful in this course?” and “What would have enabled you to be more successful in this course?”

Analyses

Pearson product-moment correlations, independent samples t tests, one-way analysis of variance (ANOVA), and ordinary least squares (OLS) regression analysis with pairwise exclusion for missing values were used to analyze the quantitative data. Collinearity testing and analyses of standardized residuals indicated that multicollinearity was not a concern, and the data contained no outliers. Thematic analysis (Braun and Clarke 2006) was used to analyze the qualitative data. Reliability was assessed by applying Cronbach’s alpha to those variables having multi-item measures and maintaining an audit trail describing in detail how qualitative data were coded and interpreted.

Results

Quantitative Results

Although the students in this sample earned relatively high grades in the course, they did not report correspondingly high levels of perceived learning. Only 9 percent of participants reported their level of perceived learning as very high, 34 percent high, 38 percent moderate, 16 percent low, and 3 percent very low. There was a moderate relationship between course grade and perceived learning.

Although less than 20 percent of participants agreed that math ability is fixed and unchangeable, about double that amount agreed with the statement: “I am just not a math person.” So, while most participants rejected the belief that in general, math ability is a fixed trait, more than one-third believed their personal math ability was limited. There were no significant gender or racial/ethnic differences in math trait belief or math self-concept. There was a moderate negative relationship between math trait belief and math self-concept.

Participants reported varying levels of anxiety on the first day of the course: 30 percent low, 31 percent moderate, 24 percent high, and 15 percent very high. Women reported significantly higher levels of anxiety than men. Table 2 presents results from the regression analyses on achievement as measured by final course grade and perceived learning. The demographic variables, gender, race/ethnicity, and major, were not significant factors predicting final course grade or perceived learning and did not improve the fit of the models and are therefore not included in the final models.

Table 2.

Regression Coefficients of Final Course Grade and Perceived Learning.

Variable	Final Course Grade			Perceived Learning
Variable	B	SE B	β	B	SE B	β
Psychodynamic:
General math trait	−.069***	.019	−.120	−.037	.061	−.019
Math self-concept	−.021	.015	−.042	.157***	.049	.094
Math ability necessary	−.097**	.034	−.083	−.507***	.112	−.129
Anxiety	−.031	.032	−.028	−.625***	.105	−.162
Utility	.023**	.007	.097	.415***	.024	.525
Cognitive
Surface learning approach	−.009	.013	−.021	−.077*	.042	−.054
Deep learning approach	−.018	.011	−.047	.167***	.037	.133
Environmental
Age	−.015**	.005	−.090	.009	.016	.016
Internal conditions
GPA	1.169***	.053	.630	.858***	.175	.137
n = 724	Adjusted r² = .503			Adjusted r² = .535

p < .05. **p < .01. ***p < .001.

Final course grade

The first model, examining final course grade, had an adjusted r² value of .503, indicating the model fits the data well, explaining half of the variance in final course grade. Factors that predicted final course grade were: math trait, math ability necessary, utility, age, and GPA. Younger students, those with higher GPAs, and those who anticipated using research methods in the future were likely to earn higher grades. The more students believed that math ability was fixed and that being good in math was necessary for understanding research methods, the less likely they were to earn higher grades.

Perceived learning

The second model, examining perceived learning, had an adjusted r² value of .535, indicating the model fits the data well, explaining a bit more than half of the variance in perceived learning. Factors that predicted perceived learning were: math self-concept, math ability necessary, anxiety, utility, surface learning approach, deep learning approach, and GPA. Students who believed they were good at math, anticipated using research methods in the future, took a deep learning approach, and had higher GPAs reported higher levels of perceived learning. Students who believed that being good at math was necessary for understanding research methods, those with higher levels of anxiety, and those who tended to take a surface learning approach reported lower levels of perceived learning.

Qualitative Results

Analysis of the responses to the open-ended questions generated the following themes.

Research culture

Students considered research separate and unfamiliar territory. Many (172) reported that prior to the course, they did not know exactly what research was and associated the term research with the natural sciences. Few were aware of any connection between research and their field of study. Students described the course as different (318) and more difficult (325) than any they had previously taken. The language was “foreign,” and the concepts were abstract, as one political science major put it: “I felt like a ten year old trying to read Shakespeare.” Many students (231) reported that in order to understand the material, they had to take a more active and engaged role “compared to being lectured to like in most normal courses.” One communications major summed it up thus: “This course was a real culture shock. It made my head spin.”

Negative hype

Because so many students experience these courses as difficult and they have such high failure and withdrawal rates, they have become shrouded in negative hype. Students had been subjected to a discourse of doom that shaped their attitudes toward these courses. This sociology student described how she felt on the first day: “I feel that all the negative publicity about the class primed me in some ways to expect failure and to have my world rocked when I stepped in.”

These responses reflect the perception that research and research methods are part of a separate, perhaps hostile, culture to which most social science students do not belong but that they must pass through in order to complete their degree.

Anxiety

Most students (454) reported feeling anxious about the course prior to taking it, and much of this anxiety continued as the course progressed. Their anxiety stemmed from multiple sources but derived mainly from the negative hype surrounding these courses. This psychology major described the source of his anxiety: “I had heard a lot about this class. It had a 50% drop/fail rate. It would make or break you in this field of study. Most people can’t handle the difficulty of research methods in psych and change majors.” Students were worried about the level of work required (336) and passing the course (378). One student described the course as “separating the real Psych majors from the fake ones.”

Many students (161) reported feeling not adequately prepared for the course, and most (412) reported having some level of math or statistics anxiety. Once the course began, students experienced continued anxiety because they were intimidated by the professor (190) and/or worried that they would not be able to understand the material (243). “A” students were particularly concerned that they would not be able to make an A in the course and their GPA would suffer.

Research self-concept

Since research was considered to be a different world, those who inhabit it were also considered different. One psychology student referred to herself as not being a member of “the nerd club.” This political science student described his instructor as being incomprehensible: “The main problem I had was with the teacher, I could hardly understand him. He seemed like he was from another planet. He just didn’t know how to translate his knowledge into what students could understand.” Additionally, the things that researchers were assumed to do both at work and in their spare time seemed peculiar to students, as this communications major related: “It is not something that normal people do, looking at a journal article with research and data. That is not the daily reading someone would do for fun, or even pick up just to kill time. I can’t imagine doing that.”

Many students (236) reported lacking the skills necessary to do research and had no desire to obtain them. Most students (434) had little interest in doing research and could not foresee any use the course might have for them. Students had conceptualized those who do research as a particular type of person, someone who was different from themselves, and someone whom they did not want to be like or could not be like. They did not see themselves as researchers; several stated, “I am not a research person” or “I am just not into research.” These comments reflect another dimension of academic self-concept, research self-concept, which represents an individual’s evaluation of his or her research ability.

Learning approach: Resistance versus leaning in

Students with weak or negative research self-concepts and those who found the course difficult reacted in one of two ways. One way was to resist learning, adopting what one student referred to as a “just get the C mindset.” Resisting behavior included not completing assigned reading, not paying attention in class, and not attending class regularly. Resisting students adopted surface learning strategies, as this communications major described: “Once you figured out how the teacher tested, the less you had to pay attention in class. I only studied what I felt I had to.” Retrospectively, many (52), like this political science major, regretted their resistant behavior: “I admit the work I did was subpar. I wish I had taken this class more seriously.”

Other students reacted by “embracing the challenge”; they “leaned in” and worked hard to succeed. Leaning in behaviors included not missing class, doing the reading, meeting with the professor, and forming study groups. These students utilized deep learning strategies such as quizzing themselves on the material and seeking out additional resources. This sociology major described her experience of leaning in:

There was so much work, a lot of which was hard for me to understand, but I continued to work at it and pushed myself to understand it. I took notes, I read as much as I could and yes, there were certain concepts that I really couldn’t grasp, but I am a hard-working student, an A student, so I was willing to go above and beyond.

Her perception of herself as a first-rate student enabled her to do what was necessary to excel in the course. Students who initially felt they were not prepared for the course succeeded because they invested time and took advantage of the resources their instructors provided.

Impact of instructor

Students attributed a significant proportion of their success or failure to their instructors. Many (67) believed success depended on which instructor they were assigned, as this political science major described: “It was my second time taking the course and this experience was much better than the first. I believe it had a lot to do with the professor who did not make an already difficult class more confusing than it needs to be.”

The most frequently reported (278) negative characteristic of instructors was “difficulty speaking English” or heavy accents. This “language barrier” made it difficult for students to understand instruction and reluctant to ask questions. Students (148) found disorganized instructors very stressful. However, many (94) successful students characterized their instructors as “great.” The attributes of great instructors included: approachability, patience, caring about students, being passionate about research, setting clear expectations, and flexibility.

Summary and Integration

Quantitative analysis indicates the following factors predict achievement as measured by perceived learning: math self-concept, the belief that being good at math is necessary for success in the course, anxiety, attributions of course utility, learning approach, and academic ability. As Illeris’s (2003, 2009) theory of adult learning suggests, these factors are distributed along the dimensions of psychodynamics, cognition, and environment. In the psychodynamic dimension, math self-concept, the belief that being good at math is necessary for success in the course, anxiety, and attributions of course utility predict perceived learning. Qualitative analysis suggests these four factors are shaped by perceptions of research as a separate culture and the negative hype surrounding the course. Qualitative analysis also suggests these factors shape students’ concepts of themselves as potential researchers, their research self-concepts.

In the dimension of cognition, quantitative analysis indicates that a deep learning approach is positively associated with perceived learning, while a surface learning approach is negatively associated with perceived learning. Qualitative analysis suggests that students’ research self-concepts inform their choice to take either a deep learning approach (leaning in) or a surface learning approach (resistance) to the course. In the environmental dimension, qualitative analysis suggests that the course instructor has an impact on perceived learning. Quantitative analysis indicates that academic ability, an internal condition of learning, also predicts perceived learning.

It is likely that the relationships between the psychodynamic factors and perceived learning are reciprocal; as the course progresses, students’ perception of learning may influence their math self-concept, belief that being good at math is necessary for success, level of anxiety, and attribution of course utility. For example, as students come to experience intermediate successes in the class, they may upwardly revise their assessment of their math ability. In a similar manner, it is likely that their learning experiences throughout the course influence their research self-concept.

Discussion and Conclusion

The purpose of this mixed methods study was to identify factors that contribute to students’ achievement in undergraduate social science research methods courses and better understand how students experience these courses. Two measures of achievement were utilized, course grade and perceived learning. Grades are an important indicator of achievement but not necessarily of learning, as indicated by the lack of a strong correlation between final course grade and perceived learning. This disparity could be a function of grade inflation, the increasing prevalence of instructors to assign relatively high grades for average work (Cole 1993), grading criteria that include such non-learning behaviors as attendance and participation, or because students have developed strategies for maximizing their grades based on grading criteria, which may be unrelated to beliefs and attitudes.

This study, grounded in adult learning theory, illustrates the usefulness of considering multiple aspects of learning. Results indicate that the psychodynamic factors anxiety, perceived course utility, belief that math ability is necessary for success, and math self-concept predict perceived learning. The finding that anxiety is negatively related to perceived learning is important given the relatively high levels of anxiety that students experience regarding the course. Perceived usefulness of the course is positively associated with perceived learning, and presumptions regarding the importance of math ability are negatively associated with perceived learning. Although students do not believe math ability is a fixed trait in general, they believe their math ability is a fixed personal trait and they are either “good at math” or “not good at math.” Students’ math self-concepts are positively associated with perceived learning. Based on their perceptions of research as a separate culture, the negative hype surrounding the course, level of anxiety, course utility, beliefs about the importance of math ability, and math self-concept, students develop a research self-concept, an appraisal of their ability to do research.

These psychodynamic factors illustrate societal attitudes reflecting a cultural divide between scientists and non-scientists identified more than 50 years ago (Snow 1961). The pervasive negative attitudes toward math can be attributed to culturally embedded conceptions that math is difficult and accessible to only a few extraordinary individuals (Belbase 2013). Restrictive images of math, science, and research are perpetuated through stories told by peers and parents, school experiences, and media representations (Hannover and Kessels 2004; Murtonen et al. 2008). These attitudes are reinforced or even exacerbated through a steady stream of negative messages about research methods courses, with sometimes exaggerated claims about difficulty, amount of work, and the caveat that success depends on proficiency in math.

Researchers have attributed students’ science aversion to the disconnect between the culture of science and students’ images of themselves (Hannover and Kessels 2004; Taconis and Kessels 2009). Through a process of self-to-prototype matching (Niedenthal, Cantor, and Kihlstrom 1985), students compare themselves to their idea of a typical scientist; those whose self-concepts do not correspond to their perceptions of scientists experience self-to-prototype mismatch. Science is largely perceived by students as dull, abstract, and hard to understand. Furthermore, successful engagement with science culture requires a “certain way of being” in addition to particular personality traits (Taconis and Kessels 2009:1130). Congruence between student self-concepts and perceptions of science culture is associated with success in math and science courses, while lack of congruence is associated with reduced interest in those courses (Lee 1998). The perceived match between self and prototype not only influences students’ affinity for the course but also their intended career choice (Hannover and Kessels 2004).

The concept of self-to-prototype matching can be extended to explain students’ attitudes toward research methods courses. Influenced by media images, societal attitudes, and personal perceptions, social science students compare their self-concepts to their idea of a typical researcher, with many students experiencing self-to-prototype mismatch as a result. I argue that research self-concept is an important previously unidentified core concept in understanding perceived learning in research methods courses.

Learning approach, a cognitive factor, predicts perceived learning. A surface learning approach (resistance) is associated with lower levels of perceived learning, while a deep learning approach (leaning in) is associated with higher levels of perceived learning. The extent of correspondence between self and prototype appears to inform students’ choice of learning approach. Students with more positive math self-concepts, those who can see themselves conducting research, and those who consider the course useful are more likely to take a deep learning approach. They lean in and adopt learning behaviors and strategies that result in higher levels of perceived learning. In contrast, students with negative math self-concepts, those averse to performing research, and those who consider the course useless adopt a surface learning approach. Since their goal is simply to pass the course, they adopt resistant learning strategies, which result in lower levels of perceived learning.

The course instructor, an environmental factor, has a substantial impact on students’ perceptions of learning. Characteristics of good instructors include being student centered, professional, and enthusiastic. Students appreciate instructors whom they feel are approachable and accessible. As far as the author can determine, no published studies specifically examine the effect instructors have on achievement in research methods courses, but research does indicate instructors’ behaviors and attitudes significantly influence undergraduate learning and engagement (Hirschy and Wilson 2002; Pfeffer and Rogalin 2012; Umbach and Wawrzynski 2005). Academic ability, an internal condition of learning, predicts perceived learning, although race/ethnicity and gender appear not to.

This study has several limitations. Participants were from a single university in the United States, limiting generalizability of the findings. The response rate was low (29 percent) but not unusual for an online survey of undergraduate students (Sax, Gilmartin, and Bryant 2003). Women and successful students were overrepresented in the sample.

Data for the psychodynamic and perceived learning variables were collected retrospectively and concurrently, although participants were asked to respond to the questions regarding math trait, math self-concept, math ability necessary, anxiety, and course utility based on their perceptions before the course began. This methodology may somewhat obscure the relationships among these variables. It is likely that students’ attitudes and perceptions change throughout the course based on instructor feedback and self-evaluation, which presents a dilemma as to when to measure the predictive variables. Rancer and colleagues (2013) measured attitudes and understanding concurrently at the end of the course. Núñez-Peña and colleagues (2013) used attitudes measured at midsemester and final grade. Sizemore and Lewandowski (2009) measured attitudes both at the beginning and at the end of the semester and found attitudes toward research did not change, but perceived utility of the course decreased. Greene and Miller (1996) first collected attitude and achievement data concurrently and retrospectively, then replicated the study collecting attitude data prior to achievement data, with no change in results. While the current study does not provide a clear picture of the dynamics of these critical variables, it does suggest plausible relationships regarding these aspects of learning.

This study has both theoretical and practical implications. Theoretically, this study provides support for Illeris’s (2003, 2009) theory of adult learning and extends the applicability of the concept of self-to-prototype matching. This study also reveals an additional dimension of academic self-concept, that of research self-concept, which encompasses one’s evaluation of one’s research ability. An important practical implication is the awareness of the impact student attitudes and beliefs have on the learning process and their potential to “undermine the efforts of methods instructors before anyone even sets foot in the classroom” (Leston-Bandeira 2013:14). Also important is the extent to which students consider instructors critical to their success.

This study suggests several opportunities for intervention: (1) Familiarize students with the process of research by integrating it more fully throughout the curriculum, (2) assess math and statistics competency at the beginning of the course and provide assistance for those needing it, (3) address anxiety at the beginning of the course by letting students know exactly what will be expected of them, (4) institute a prerequisite course that teaches/reviews the skills necessary for success, (5) explain to students the benefits of taking a deep learning approach to the course and encourage them to engage more fully with coursework instead of relying on memorization, and (6) create a faculty learning community in which the specific skills necessary for teaching research methods are developed and shared.

This study highlights the importance of psychodynamic factors and reveals a new factor grounded in the qualitative data, research self-concept, which suggests how the psychodynamic and cognitive dimensions of learning may be linked. Future research should include quantifying those concepts identified through qualitative analysis and developing a fully quantitative model. This study is important because it increases our understanding of the factors that influence student achievement in social science research methods courses. A better understanding of these factors could facilitate interventions leading to improvements in student success and program retention and graduation rates.

Footnotes

Editor’s Note

Reviewers of this manuscript were, in alphabetical order, Jessica Crowe, Carla Pfeffer, and Ann Prince.

Funding

The author disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was partially supported by grants from the College of Humanities and Social Sciences and the Center for Excellence in Teaching and Learning, Kennesaw State University.

Author Biography

Gail Markle is an associate professor of sociology in the Department of Sociology and Criminal Justice at Kennesaw State University in Kennesaw, Georgia. Dr. Markle’s research interests include pro-environmental behavior, the scholarship of teaching and learning, and campus diversity issues.

References

Belbase

Shashidhar

. 2013. “Images, Anxieties and Attitudes toward Mathematics.” International Journal of Education in Mathematics, Science and Technology 1(4):230–37.

Biggs

John B.

1988. “Approaches to Learning and Essay Writing.” Pp. 185–229 in Learning Strategies and Learning Styles, edited by Schmeck

R. R.

New York: Plenum Press.

Biggs

John B.

Kember

David

Leung

Doris

. 2001. “The Revised Two Factor Study Process Questionnaire: R-SPQ-2F.”British Journal of Educational Psychology 71(1):133–49.

Blalock

H. M.

Jr.

1987. “Some General Goals in Teaching Statistics.”Teaching Sociology, 15(2):164–72.

Bos

Angela L.

Schneider

Monica C.

2009. “Stepping around the Brick Wall: Overcoming Student Obstacles in Methods Courses.”PS: Political Science and Politics 42(2):375–83.

Braun

Virginia

Clarke

Victoria

. 2006. “Using Thematic Analysis in Psychology.”Qualitative Research in Psychology 3(2):77–101.

Burkley

Melissa

Parker

Jessica

Stermer

S. Paul

Burkley

Edward

. 2010. “Trait Beliefs That Make Women Vulnerable to Math Disengagement.”Personality and Individual Differences 48(2):234–38.

Cole

William

. 1993. “By Rewarding Mediocrity, We Discourage Excellence.”The Chronicle of Higher Education 6:B3–B4.

Creswell

John W.

Clark

Vicki L. Plano

. 2011. Designing and Conducting Mixed Methods Research. 2nd ed. Thousand Oaks, CA: Sage.

10.

Crowe

Jessica A.

Silva

Tony

Ceresola

Ryan

. 2015. “The Effects of Peer Review on Student Learning Outcomes in a Research Methods Course.”Teaching Sociology 43(3):201–13.

11.

Diseth

Âge

Martinsen

Øyvind

. 2003. “Approaches to Learning, Cognitive Style, and Motives as Predictors of Academic Achievement.”Educational Psychology 23(2):143–55.

12.

Dweck

Carol S.

Chiu

Chi-Yue

Hong

Ying-Yi

. 1995. “Implicit Theories and Their Role in Judgments and Reactions: A World from Two Perspectives.”Psychological Inquiry 6(4):267–85.

13.

Earley

Mark A.

2014. “A Synthesis of the Literature on Research Methods Education.”Teaching in Higher Education 19(3):242–53.

14.

Greene

Barbara A.

Miller

Raymond B.

1996. “Influences on Achievement: Goals, Perceived Ability, and Cognitive Engagement.”Contemporary Educational Psychology 21(2):181–92.

15.

Grove

Wayne A.

Wasserman

Tim

Grodner

Andrew

. 2006. “Choosing a Proxy for Academic Aptitude.”Journal of Economic Education 37(2): 131–47.

16.

Hannover

Bettina

Kessels

Ursula

. 2004. “Self-to-prototype Matching as a Strategy for Making Academic Choices: Why High School Students Do Not Like Math and Science.”Learning and Instruction 14(1):51–67.

17.

Hirschy

Amy S.

Wilson

Maureen E.

2002. “The Sociology of the Classroom and Its Influence on Student Learning.”Peabody Journal of Education 77(3):85–100.

18.

Illeris

Knud

. 2003. “Towards a Contemporary and Comprehensive Theory of Learning.”International Journal of Lifelong Education 22(4):396–406.

19.

Illeris

Knud

. 2009. “A Comprehensive Understanding of Human Learning.” Pp. 7–20 in Contemporary Theories of Learning: Learning Theorists in Their Own Words, edited by Illeris

NY: Routledge.

20.

Lee

James Daniel

. 1998. “Which Kids Can ‘Become’ Scientists? Effects of Gender, Self-concepts, and Perceptions of Scientists.”Social Psychology Quarterly 61(3):199–219.

21.

Leston-Bandeira

Cristina

. 2013. “Methods Teaching through a Discipline Research-oriented Approach.” Politics 33(3):207–19.

22.

Macheski

Ginger E.

Lowney

Kathleen S.

Buhrmann

Jan

Bush

Melanie E.L.

2008. “Overcoming Student Disengagement and Anxiety in Theory, Methods and Statistics Courses by Building a Community of Learners.” Teaching Sociology 36(1):42–48.

23.

Marsh

Herbert W.

1993. “The Multidimensional Structure of Academic Self-concept: Invariance over Gender and Age.”American Educational Research Journal 30(4):841–60.

24.

Marsh

Herbert W.

Seaton

Marjorie

. 2013. “Academic Self-concept.” Pp. 62–63 in International Guide to Student Achievement, edited by Hattie

Anderman

E. M.

New York: Routledge.

25.

Murtonen

Mari

Lehtinen

Erno

. 2003. “Difficulties Experienced by Education and Sociology Students in Quantitative Methods Courses.”Studies in Higher Education 28(2):171–85.

26.

Murtonen

Mari

Oklinuora

Erkki

Tynjälä

Päivi

Lehtinen

Erno

. 2008. “‘Do I Need Research Skills in Working Life?’: University Students’ Motivation and Difficulties in Quantitative Methods Courses.”Higher Education 56(5):599–612.

27.

Niedenthal

Paula M.

Cantor

Nancy

Kihlstrom

John R.

1985. “Prototype Matching: A Strategy for Social Decision Making.”Journal of Personality and Social Psychology 48(3):575–84.

28.

Núñez-Peña

M. I.

Súarez -Pellicioni

Bono

2013. “Effects of Math Anxiety on Student Success in Higher Education.”International Journal of Educational Research 58(1):36–43.

29.

Onwuegbuzie

Anthony J.

Wilson

Vicki A.

2003. “Statistics Anxiety: Nature, Etiology, Antecedents, Effects, and Treatments: A Comprehensive Review of the Literature.”Teaching in Higher Education 8(2):195–209.

30.

Parker

Jonathan

Dobson

Andy

Scott

Sue

Wyman

Matthew

Landén

Angelika

. 2008. International Bench-marking Review of Best Practices in the Provision of Undergraduate Quantitative Methods in the Social Sciences. Staffordshire, UK: Keele University.

31.

Parks

Malcolm R.

Faw

Meara

Goldsmith

Daena

. 2011. “Undergraduate Instruction in Empirical Research Methods in Communication: Assessment and Recommendations.”Communication Education 60(4):406–21.

32.

Pfeffer

Carla A.

Rogalin

Christabel L.

2012. “Three Strategies for Teaching Research Methods: A Case Study.”Teaching Sociology 40(4):368–76.

33.

Rancer

Andrew S.

Durbin

James M.

Lin

Yang

. 2013. “Teaching Communication Research Methods: Student Perceptions of Topic Difficulty, Topic Understanding, and Their Relationship with Math Anxiety.”Communication Research Reports 30(3):242–51.

34.

Richardson

Frank C.

Suinn

Richard M.

1972. “The Mathematics Anxiety Rating Scale: Psychometric Data.”Journal of Counseling Psychology 19(6):551–54.

35.

Sax

Linda J.

Gilmartin

Shannon K.

Bryant

Alyssa

. 2003. “Assessing Response Rates and Nonresponse Bias in Web and Paper Surveys.”Research in Higher Education 44(4):409–32.

36.

Sizemore

O. J.

Lewandowski

Gary W.

Jr.

2009. “Learning Might Not Equal Liking: Research Methods Course Changes Knowledge but Not Attitudes.” Teaching of Psychology 36(2):90–95.

37.

Snow

C. P.

1961. The Two Cultures. New York: Cambridge University Press.

38.

Stalp

Marybeth C.

Grant

Linda

. 2001. “Teaching Qualitative Coding in Undergraduate Field Methods Classes: An Exercise Based on Personal Ads.” Teaching Sociology 29(2):209–18.

39.

Taconis

Ruurd

Kessels

Ursula

. 2009. “How Choosing Science Depends on Students’ Individual Fit to ‘Science Culture.’” International Journal of Science Education 31(8):1115–32.

40.

Tan

JooEan

Yiu-Chung

. 2004. “Using Feature Films to Teach Observation in Undergraduate Research Methods.” Teaching Sociology 32(1):109–18.

41.

Taylor

Ralph B.

McConnell

Patrick

. 2001. “BAC and Beer: Operationalizing Drunk Driving Laws in a Research Methods Course Exercise.” Teaching Sociology 29(2):219–28.

42.

Tobias

Sheila

. (1993). Overcoming Math Anxiety. New York: W.W. Norton & Company.

43.

Umbach

Paul

Wawrzynski

Matthew R.

2005. “Faculty Do Matter: The Role of College Faculty in Student Learning and Engagement.” Research in Higher Education 46(2):153–84.