Abstract
Sociology is a discipline in which the idea of a multi-method research design has held credence for many years, far before the term mixed methods research was coined. This article charts the implementation and framing of this approach over time to better understand the place and state of mixed methods research in the discipline today. Several recent applications of mixed method research by sociologists are highlighted to demonstrate the range of projects being conducted. There are challenges to further development of mixed methods inquiry within Sociology; however, the current epistemological base of the approach—pragmatism—promotes the merits of a variety of theoretical and methodological approaches and the discipline is better for it.
The idea that an empirical research endeavor could capitalize on the use of both qualitative and quantitative data, and benefit from the integration of a varied set of methods has held currency in sociology for decades. The rapid increase across all of social science in attention toward and codification of this approach, now labeled mixed methods research, 1 provides an opportunity for reflection. In this article, I review (albeit cursorily) the history of methodological approaches and tendencies in Sociology over the last century or so, describe current applications of and advances in mixed methods research in the field, and examine outstanding issues and future directions for mixed methods inquiry in sociology.
Road to the Present
In the selection of a research method, forces of structure and agency shape the outcome. Although we might like to think social scientists select the absolutely perfect research design and method(s) to investigate a given question, we know that agency has its limits, and research decisions are also influenced by one’s methodological training (e.g., the availability and quality of training in various methods) and the proclivities of one’s audiences (be they journal or book reviewers, senior faculty, tenure and promotion letter writers, or funders; Denscombe, 2008). These structural forces shape and are shaped by reigning methodological paradigms, or sets of widely accepted beliefs and values about how research should be conducted, that shift through time (Bryman, 2004). Three commonly assumed, if not directly referenced, paradigms of research in Sociology have been the so-called qualitative or constructivist paradigm, the quantitative or positivist paradigm, and the newly developing pragmatic paradigm.
Up until the first half of the 20th century, methods described as being “qualitative” or “fieldwork” (Sieber, 1973), such as participant observation, semi-structured interviewing (especially in a life history interview format), and content analysis of archival materials, were dominant in Sociology (e.g., DuBois, 1995; Thomas & Znaniecki, 1918). Sociologists started taking the principles of Weber and Simmel’s interpretive understanding, or verstehen 2 and systematizing social observation into a variety of data collection methods. Although one might argue that before the mid-20th century Sociology involved primarily theoretical or “qualitative” research. A mix of methods was characteristic from early on, many of them being pioneered within the Chicago School. University of Chicago’s early sociologists studied the city with ethnographic, archival, and survey methods (Blumer, 1984). Students of the Chicago School, such as E. Franklin Frazier, parlayed the approach to their own areas of interest. Frazier (1932) used qualitative and quantitative data accumulated from personal documents to describe and interpret African American family life early in the 19th century.
In addition to transforming qualitative data into quantitative data (called quantizing qualitative data in the mixed methods literature), studies in the first half of the 20th century also began to incorporate structured, closed-ended questions and probability or representative sampling in their studies, often leading to the production of original survey data for analysis. For example, in addition to observation and semi-structured interviewing, the cutting edge Middletown studies in the 1920s and 1930s involved survey questionnaires administered to select groups of the town’s population, such as samples of high school seniors, housewives, and religious clergy (Caplow, Bahr, & Chadwick, 1983; Lynd & Lynd, 1929, 1937).
In their famous study of an international typographical union, whose clearly democratic ways seemed to contradict Michel’s (1915) Iron Law of Oligarchy, Lipset, Trow, and Coleman (1956) used a mix of less structured interview and structured survey methods, producing data analyzed as both words and numbers to elicit findings that helped to revise theory about organizational structures. In the research resulting in Boys in White (Becker, Geer, Hughes, & Strauss, 1961), investigators primarily relied on participant observation of students and faculty at one medical school, but incorporated semi-structured interviews of systematic samples of these students and faculty. These in-depth case studies represent the rich history of multi- and often mixed methods research existing in sociology. 3
Following World War II, there was an explosion in large-scale survey research (Platt, 1996; Sieber, 1973). The technical advances in survey data collection, measurement, and analysis, developed alongside a strong desire for many social scientists to achieve respect and status as “scientists” and the height of the so-called “positivist” era. As the ability to collect standardized data and test hypotheses in the spirit of the scientific method[s] proliferated, the U.S. economy flourished, government and private funders grew enamored of survey-based research, and polling for political and marketing purposes grew in popularity, the survey industry thrived. 4 The balance of power in the discipline tipped toward sociologists collecting and/or analyzing survey data, and sociological training and the content of the discipline’s journals increasingly tilted toward a dominance of the quantitative paradigm (Morgan, 2007). Sociologists also became analysts of secondary survey data, and a smaller proportion invested time and energy in personal observations and involvement with the cases under study (Axinn & Pearce, 2006). Further, a strong reliance on the language and protocol of what is called “the scientific method,” albeit more in presentation than application, portrayed a strong commitment to positivist ideals. As this new style of research gained prominence, there arose a tension between its converts and those who favored the older field-based, case study, less structured methods; this developed into a strong tension between those in support of the “deep, rich” data approach and those professing the superiority of “hard, generalizable” data (Sieber, 1973). This polemic developed into strong subcultures and the dualism known as the paradigm wars between the “qualitative” and “quantitative” camps (Bryman, 1984; Hanson, 2008; Smith, 1991).
Through the 1960s, 70s, and 80s, there was a resurgence in support for “qualitative” methods, “naturalistic inquiry” (Lincoln & Guba, 1985), or “constructivism” in an effort to create an alternative for what many saw as the failure of “positivism.” The constructivist, or “qualitative” approach, was built upon the legacy of “interpretive sociology” and “symbolic interaction,” and it paralleled developments in postmodern theory (Hanson, 2008). Morgan (2007) labels this resurgent paradigm the metaphysical paradigm, because its emergence has been grounded in ontological issues. This paradigm is also recognized for its focus on the role of the researcher and reflexivity in the research process (Kleinman, 2007; Kleinman & Copp, 1993). Going forward, I will rely on Morgan’s term—metaphysical paradigm—because I object to the use of the word qualitative (and quantitative) in reference to anything other than data. Calling a paradigm, a method, or an approach qualitative or quantitative is overly reductionist and counterproductive to a holistic understanding of research as a process (Axinn & Pearce, 2006).
The metaphysical paradigm is centered on the assumption that there is no one, true reality; therefore, attempts at objective measurement are futile. This is generally posed against what is argued to be the positivist, or more “quantitative,” paradigm that assumes reality exists and can be measured. Proponents of the metaphysical paradigm often assert that the two paradigms create incompatible kinds of knowledge, but this argument fails on multiple levels (Morgan, 2007) and disregards the practical considerations and many successes of actual research (Axinn & Pearce, 2006; Hanson, 2008). As Morgan (2007) and others have convincingly argued, the qualitative–quantitative or metaphysical–positivist duality is an overly rigid dichotomy, or as Lieberson (1992) calls it, “a Durkheimian conflict,” in that caricatures are made of the research that “others” conduct in order to make a statement about one’s own type of research. In other words, although it has come to be accepted that research cannot produce social facts that reduce social action to universal laws, a sociology based on the notion that no commensurable realty exists is futile as well (Morgan, 2007).
Throughout the late 20th century, a handful of sociologists have made strong arguments to reconsider the polemic frame of qualitative versus quantitative research, starting with Sieber (1973), then Bryman (1984), Denzin (1989), and Smith (1991). Now with the explosion of interest in mixed methods research, other sociologists have begun to further articulate the problems with dividing the world of sociological research into qualitative and quantitative sides (Axinn & Pearce, 2006; Hanson, 2008). What is resulting from the rethinking of the epistemology undergirding our research choices (in a variety of fields) is the birth of a third paradigm often called the “pragmatic approach” (Creswell, 2003; Morgan, 2007). Morgan (2007) articulately outlines this middle ground where the seemingly deterministic ways of positivism and the over-deconstruction of the metaphysical approach are balanced by a focus on lines of action (William James and George Herbert Mead), warranted assertions (John Dewey), and workability (James and Dewey). This is akin to what Stanley Lieberson (1992) argued for as better attention to the probabilistic nature of human or social behavior instead of deterministic language and hypotheses or the abandonment of quantitative analyses.
Morgan (2007) proposes three specific ways in which the pragmatic approach goes above and beyond either the qualitative/metaphysical approach or the quantitative/positivist approach, transcending the counterproductive dualism that has been characteristic of sociology. First, many scholars would describe the process of theory development in “qualitative research” as being very inductive (from specific cases and observations to transferable theory) and “quantitative research” as being very deductive (from theory to hypotheses tested with representative cases). However, few so-called qualitative research projects start without any sense of a research question or theoretical background, and few so-called quantitative projects move in one direction from a theory to a hypothesis to a test and call it quits (Smith, 1991). In reality, research of all types makes several moves back and forth between theory (re)construction and data analysis (Axinn & Pearce, 2006; Hanson, 2008). Therefore, as Morgan (2007) puts it, a pragmatic approach calls for abduction, the complementary and constant dialectic between inductive and deductive theoretical development rather than a reliance on one or the other.
Second, a hallmark of the metaphysical paradigm is the subjectivity researchers bring to their work, and the assumed goal of a quantitative approach is to maintain as much objectivity as possible. It is impossible to attain pure subjectivity or objectivity, and the pragmatic approach more explicitly recognizes this. Morgan (2007) calls it intersubjectivity when a researcher or research team works back and forth between rigorous attempts at both subjective and objective frames (Hanson, 2008; Morgan, 2007). In essence, the pragmatic approach accepts that there is both a single “real world” and that all humans have their own unique interpretations of that world (Lieberson, 1985; Smith, 1991). It is up to sociologists to follow the call of Bourdieu and Wacquant (1992) and turn on ourselves the scientific weapons we produce in a truly reflexive sociology. A reflexive sociology is one in which we recognize how the actions, understandings, and interpretations of a researcher influence the people and settings they are studying and the findings they produce (Thomas, 1923). One such line of research embracing the goals of intersubjectivity and reflexivity are those sociologists and others taking an intersectional approach. As Perry (2009, pp. 235–236) argues:
The traditions of intersectionality and mixed methods research have a shared commitment to holistic inquiry . . . are equally committed to assessing the interplay between human agency and systemic structures, and center the tensions and contradictions of our lived experiences in their assessment of social and behavioral realities.
A third way that the pragmatic paradigm clarifies tension between earlier paradigms is the recognition that all forms of data can be “transferrable,” or leads to the development of arguments for how our discoveries translate to other social moments and settings (Lincoln & Guba, 1985; Morgan, 2007). This is to ease the apprehension about the metaphysical paradigm’s assumed limitation of inference to the precise context of study versus the “positivist” approach to be more generalizable to the population of focus. In reality, it is unlikely that a study in one specific setting has no applicability beyond that setting, nor can one study be generalized to every possible historical or cultural setting. In other words, we should evaluate each study for its specific contributions and add it to the empirical record toward answering larger questions (Small, 2009).
Although Morgan (2007) names and describes the emergent pragmatic paradigm explicitly to provide justification for mixed methods research, acceptance of this paradigm does not require one to commit to the practice of mixed methods research. It only requires a pluralistic approach, or the acceptance that no one set of research tools is categorically better than another, and encourages us to acknowledge the strengths in a variety of methods and approaches. This is what Bourdieu (2004) encourages as “methodological polytheism” (p. 101) manifested in his work as statistical analyses combined with direct observation (Fries, 2009; Wacquant, 1998). Also, if one prefers to specialize in a particular type of research method, she or he should at least commit to respect for and familiarity with the research methods and findings others use to bring evidence to bear on a research problem or question. As Lieberson (1992) argues, sociological research is at its best when we rely on “building blocks of evidence,” or numerous forms of evidence from a variety of perspectives and methods. Whether a scholar chooses to create multiple building blocks from different methods in his or her own research, or to contribute to the larger process through mono-method research that is supplemented by other’s research that uses different methods, is up to each researcher. What seems essential to the new paradigm is a willingness to see research methods and designs as a set of tools that should be selected and evaluated based on their own specific practical merits (e.g., level of structure, involvement of investigator, type of data produced), not any connection to the so-called quantitative or qualitative research approaches (Axinn & Pearce, 2006).
How far has this third paradigm of “pragmatism” come in sociology? It is hard to say. It is still common to overhear (in the introduction of one sociologist to another) the question, “Are you quantitative or qualitative?” as if whole research careers can be reduced to one or the other. From a pragmatic viewpoint, it is more correct to apply “the Q words” to data, and identify our professional identities, writing, courses, and research projects by specific topic areas, theoretical frames, or research methods (e.g., surveys, semi-structured interviews, participation) instead. A promising sign of the emergence of this pragmatic paradigm is the growing number of studies and publications that achieve integration of qualitative and quantitative data in a quest to answer key sociological questions. Below I outline a few recent examples.
Current Applications of Mixed Methods Inquiry in Sociology
There are many design options for mixed methods research, as nicely summarized by Creswell and Plano Clark (2007) and Tashakkori and Teddlie (1998). As the field of mixed methods research gains increasing attention, and as its concepts, designs, and epistemology develop, the related terms have come into greater use in many disciplines (Denscombe, 2008; Johnson, Onwuegbuzie, & Turner, 2007; Yoshikawa, Weisner, Kalil, & Way, 2008), but, surprisingly, not as quickly in sociology as in others, especially considering this revived and renewed approach is based in the long history of multimethod 5 research in sociology and cultural anthropology (Johnson & Gray, 2010; Creswell, 1999; Johnson et al., 2007).
A search of the top three general journals, American Journal of Sociology, American Sociological Review, and Social Forces, and the top two sociological methods journals, Sociological Methodology and Sociological Methods and Research, between 1990 and 2010, reveals only one research article (Laubach, 2005, summarized below), one published American Sociological Association (ASA) Presidential Address, and one book review that use the term mixed method* or mixed-method* anywhere in the text. A search for articles using both qualitative data analysis and quantitative data analysis produces 27 articles in this time period for these five journals. It is likely that other mixed methods articles are not recognized in these searches because of their reliance on terms other than mixed methods or qualitative and quantitative.
One might ask why sociology has been slow to embrace the newly minted terms associated with mixed methods research or to cite the increasingly systematized guides and designs available in a growing literature on the conduct of mixed methods research. There are a variety of possible answers that probably all partially explain this response. For one, as described earlier, there is a long history of mixing research methods in sociology, especially for in-depth case studies, so many sociologists do not necessarily feel a need to justify the use of multiple methods with the newer methodological terms and guidebooks. Second, because the field remains in a transition from the duality of the quantitative-versus-qualitative paradigm to the pluralistic underpinnings of the pragmatic paradigm, there is still a tendency to avoid controversial language in article manuscripts. Students are often coached to rely on the technical details of the methods and data used, rather than employing too much language about qualitative, quantitative, or mixed methods research that could ruffle the feathers of reviewers committed to certain conceptualizations of research styles and epistemologies. One does not want to seem to be overselling the value of a project based on the general richness of a mixed methods approach to the detriment of explaining the quality of each specific methodological tool in use.
Regardless of the labels researchers apply to their work, there are many outstanding examples of mixed methods research being conducted by sociologists. The examples come in many forms, ranging in size, the balance of methods, whether any or all of the data are primary or sourced from other projects, and how the integration of methods is achieved. Below I outline several exemplars of current styles of mixed methods inquiry in sociology and discuss how they nicely demonstrate the achievement of abduction, intersubjectivity, and transferability as described by Morgan (2007).
The Three-City Study
One example of a large-scale, multi-method research project run by a multidisciplinary team of principal investigators (many of whom are sociologists) that has led to a series of interesting mixed methods analyses, is the Three-City Study (e.g., Cherlin, Burton, Hurt, & Purvin, 2004; Cherlin, Cross-Barnet, Burton, & Garrett-Peters, 2008). This was a study conducted in Boston, Chicago, and San Antonio to assess the well-being of low-income children and families in the post–welfare reform era. The study began in 1999 and involves three interrelated components: a longitudinal survey, embedded developmental studies with a subsample of the survey respondents, and contextual, comparative ethnographic studies with families from the same neighborhoods from which survey respondents were sampled. Surveys were conducted in 1999, 2001, and 2005 with a focal child and caregivers, developmental studies were conducted in 1999 and 2001, and families were recruited into the ethnography between June 1999 and December 2000 and visited by an ethnographer approximately once or twice per month for 12 to 18 months and then every 6 months thereafter through 2003.
The Three-City Study Survey was first conducted in 2,400 systematically sampled households from low-income neighborhoods in all three cities with a resident child between the ages 0 and 4 or 10 and 14 in 1999. These same families were followed up for surveys in 2001 and 2005, with 80% of focal children who participated in the 1999 survey participating again in 2005. To improve the breadth and depth of the child evaluations included in the survey part of the study, the Three-City Study included an embedded development study (EDS) with children from ages 2 to 4 at Wave 1 of the longitudinal survey. The EDS component incorporated videotaping and coding of caregiver-child interactions, time-diary studies, and observations of child care settings. Further, there was also an ethnographic component of the Three-City Study, involving 256 additional families, recruited nonrandomly, who were not in the survey sample, but who resided in the same neighborhood from which survey respondents were sampled.
As this study shows, one challenge facing mixed methods studies that incorporate participant observation is being able to involve the same participants who participated in the surveys in the fieldwork component. Participant observation is so time intensive and invasive for participants that it is difficult to recruit them in the same way we recruit survey respondents and achieve reasonable response rates. Instead, a researcher conducting participant observation needs to slowly gain legitimization in a community and earn the trust of participants. Therefore, the design of the ethnographic component of the Three-City Study involved gaining entry to the community through local organizations and not requiring that fieldwork be conducted with survey respondents. Recruitment sites included formal child care settings (e.g., Head Start), the Women, Infants, and Children (WIC) program, neighborhood community centers, local welfare offices, churches, and other public assistance agencies.
To gather ethnographic data about these 256 families, the method of structured discovery was used, in which in-depth interviews and observations were focused on specific topics but allowed flexibility to capture unexpected processes (Burton et al., 2001; Winston et al., 1999). Interviews were conducted with the biological or adoptive mother or primary caregiver of the focal children, and team ethnographers engaged in participant observation with the family, which often involved accompanying the mother and her children to the welfare office, doctor, grocery store, or workplace, and taking note of the interactions and contexts of those places. These ethnographies provide a rich contextualization of the communities and households in which the accompanying survey data were collected.
One example of the kind of mixed methods research coming out of this exceptional study is an article by Cherlin et al. (2004) about how women’s experiences of physical and emotional abuse are related to their subsequent relationship patterns. This article is an excellent exemplar of mixed methods research, appearing in a leading sociology journal. The authors first analyzed the survey data using frequency tables and cross-tabulations to examine the relationship between patterns of union formation and reported experiences of physical and sexual abuse. They then conducted a comprehensive grounded theory analysis using the complete set of ethnographic data to categorize patterns of abuse and union formation that hugely extend the field’s understanding of connections between past abuse and future relationship forms. These ethnographic findings were then used to develop hypotheses that were tested with the survey data. For example, the ethnographic analysis suggested that the timing of abuse (i.e., whether it occurred in childhood, adulthood, or in both periods) and the form of abuse (i.e., whether it was sexual or physical) were strongly related to distinctive union patterns (e.g., repeated short-term and often abusive relationships or complete avoidance of relationships). The authors then tested and confirmed theory generated from the relationships between abuse and union formation identified in the ethnographic analysis using multivariate statistical models of the survey data. The nice, iterative way in which the research was presented provides an excellent example of the abduction, or constant interplay between inductive and deductive reasoning, possible to push theory and analyses of a particular social problem forward.
Study of Gender, Migration, and HIV Risk
Another example of a sociological study employing a mix of methods to collect and analyze multiple forms of data is a study of gender, migration, and HIV risks among Hispanic migrants in Durham, North Carolina (Parrado, McQuiston, & Flippen, 2005). This project used a multi-method strategy for studying difficult-to-reach migrant populations that combines community collaboration with survey methods. There are many complications to collecting a random sample of migrants, including the unavailability of a sampling frame of foreign-born U.S. residents, and the challenges to getting cooperation from undocumented migrants who fear deportation and detention. To overcome these barriers, as well as the difficulties in asking questions about sexual behavior and gender dynamics in marginalized communities, the research team combined methods often reserved for more intensive, community-based ethnography to develop a rich familiarity with the context, norms, and culture surrounding the population under study. The study is based on a form of study design called a micro-demographic community study (Axinn, Fricke, & Thornton, 1991; Massey, 1987), which employs less-structured, intensive methods (e.g., participant observation and semi-structured interviewing) to overcome challenges in sampling, survey design, and the interpretation of quantitative analyses.
Community-based participatory research (CBPR) is a collaborative approach that involves community members in all stages of the research process, enhancing the understanding of observed social behaviors. This research approach is well suited for the integration of knowledge and action needed to improve the health and well-being of local communities (Cheatham & Shen, 2003). In this particular case, community involvement and less structured research methods were key to gaining access to a difficult-to-reach population, developing a flexible survey instrument, increasing data quality, and grounding findings within the cultural realities of local migrants.
To form the community group that participated in all phases of study design and data collection, the researchers drew from 8 years of both formal and informal involvement with the local Hispanic community, including focus groups and an HIV prevention lay health advisory program conducted in collaboration with a local grassroots organization (McQuiston, Choi-Hevel, & Clawson, 2001; McQuiston & Flaskerud, 2003), to recruit community members who had interests in working on HIV prevention among migrants. This resulted in 14 community members, 6 men and 8 women, who were integral to all aspects of the research process, including the conceptualization of the project, execution of the data collection, and interpretation of research findings.
Through a process of group discussion, the CBPR team helped identify key elements of social adaptation that were central to HIV risk, including social isolation, alcohol consumption, and commercial sex worker use, serving the dual purpose of bringing locally specific information and context to the research design and also giving community members greater ownership of the investigative process. In addition, CBPR members were an invaluable resource in identifying areas of geographic concentration of Hispanics and in addressing gender differences in how to reach respondents. Their insights informed the random sampling design and strategy for the data collection stage (Parrado et al., 2005). For example, one group member described how he would ask immigrants if they knew about the Individual Tax Identification Number (ITIN) system as a segue into asking if they had legal documentation.
Collaboration between the academic research team and the CBPR participants resulted in a successfully flexible semi-structured questionnaire, or hybrid data collection tool (Axinn & Pearce, 2006), that allowed for the reconstruction of individual experiences (e.g., complex migration histories) while maintaining a casual conversation with the respondent. The conversational style was particularly important during the opening sections of the survey, which collected information on social background and family status. The trust and rapport established during these opening sections allowed for more open discussion of sensitive issues pertaining to gender norms and sexual behaviors, including sexual initiation, use of commercial sex workers, and extramarital partners.
Parrado et al. (2005) also report that CBPR was invaluable for the analysis and interpretation of the survey data. CBPR members recorded their observations as field notes at the end of each survey, describing local conditions and any other interesting material not recorded in the questionnaire. This provided invaluable information about the context of the various apartment complexes, including public drinking, police presence, and institutional aspects of the commercial sex industry. Through interactions and unstructured discussions with study participants, the CBPR team members were able to glean significant information about how the sex industry is advertised, promoted, and accessed by local residents. After survey data collection was completed, CBPR members’ insights and observations were brought to bear on data analysis through regular and ongoing presentation of quantitative findings, discussions of them, and then refinements of the analyses. All in all, as demonstrated in this ambitious project, combining CBPR methods with survey methods achieves more of a balance in the goals of subjectivity and objectivity that Morgan (2007) labels intersubjectivity than any one set of methods could achieve. This project is an excellent example of Bourdieu’s (2004) recommendation that sociologists exercise epistemological vigilance, or the constant clarification and (re)formation of propositions and hypotheses so they are empirically verifiable. This ferrets out personal bias through conscious investigation of the cultural field that structures intellectual orientations and reveals the context that can bias social research (Bourdieu, 2004).
National Study of Youth and Religion (NSYR)
An example of a research study designed to allow for the integration of two types of data during the analysis phase is the National Study of Youth and Religion. This project includes a longitudinal survey of a random digit dial sample of youth ages 13 to 17 in 2002 with follow-up surveys in 2005 and 2007 together with semi-structured, in-person interviews conducted with a subsample of youth survey respondents at all three waves of the study. This project is designed to track the religious and spiritual lives of American youth from adolescence through the transition to adulthood, and aims to balance the structured, systematic data collected by the survey with the more nuanced, detailed, qualitative data that come from the in-person interviews.
Three books written by members of the NSYR research team rely on iterative analyses of the survey and semi-structured interview data to describe the contours of religiosity and spirituality in youth as well as the changes in religious identity over time (Pearce & Denton, 2011; Smith & Denton, 2005; Smith & Snell, 2009). All three books aim to achieve the transferability proposed by Morgan (2007) by moving back and forth between the rich detail achieved through the narratives of youth themselves and checking for statistical generalizability in the nationally representative survey data. It is in the merging of the survey data patterns and the voices of youth that a truly rich and interesting picture of youth religiosity and spirituality emerges. For example, Smith and Denton (2005) reveal somewhat surprisingly high levels of religious belief and involvement among teenagers as measured by standard survey questions, but they contextualize these survey findings in rich description of just how youth view their faith and practice culled from the in-person interviews. In their book, Smith and Denton (2005) demonstrate a strong tendency of youth to reference what Smith and Denton call moral therapeutic deism, or the belief in a creator God who watches over human life on earth, wants people to be good, nice, and fair to each other, but does not need to be particularly involved in one’s life except when needed to resolve a problem.
Pearce and Denton (2011) reveal small decreases in religiosity over the 3 years between Waves 1 and 2 of the study apparent in both the survey and the semi-structured interview data, but the interviews (or qualitative data) are crucial in interpreting what this change means to adolescents, because when asked if they themselves think their religiosity is increasing, decreasing, or staying the same, the youth are mostly likely to say it stayed the same and least likely to say it decreased. The interviews reveal that during adolescence, youth take ownership of their beliefs and practices, so even if they wane in importance, youth feel equally or more religious than before. In both of these cases of analysis, the survey and interview data are key in providing a holistic picture of a phenomenon that may be misinterpreted, risking its transferability, without the mutually supportive analyses of survey and semi-structured interview data.
These three examples of current-day, mixed methods research within sociology, capitalizing on the approach to deliver upon the key features of the newly emerging pragmatic paradigm as defined by Morgan (2007), are large-scale, multi-investigator, well-funded, and primary data collection studies. There are many other high quality examples of mixed methods applications in sociology conducted by one or two scholars that are not always characterized by all forms of data being collected by the investigators themselves (e.g., they rely on the analysis of secondary data for one aspect of the study). These models are more widely replicable and are often used for dissertation research in sociology today, suggesting a growing acceptance of the pragmatic paradigm among budding young sociologists despite most graduate programs’ methodological training still being organized around an overly rigid dichotomy between qualitative and quantitative approaches.
One example of an article resulting from an impressive combination of investigator involved ethnography mixed with secondary survey data analysis is Laubach’s (2005) use of ethnography in an organization’s workplace to develop a theoretical model of stratification in the workplace, showing its connection to job rewards such as wages. He then developed survey measures of his key concepts and tested his model. The survey results confirmed his inductive, data-based revisions to previously existing theory on consent within informal organizations. The mix of his ethnographic and survey findings reveals the power of mixed methods for advancing a field with relatively codified scales and instruments to measure a set of concepts that had come to be accepted as complete.
Another example of a mixed methods investigation designed to answer a focused question is the research presented by Smith and Moore (2000), which examines the perceptions of closeness among the African American students at a primarily white university. These researchers used a survey of all African American students at the university and follow-up semi-structured interviews with a subsample of the survey respondents designed to represent four key groups (those who reported a multiracial background, those whose friends were all African American, those who reported half or some of their friends to be African American, and those who reported few or none of their friends to be African American). The same patterns in the levels of closeness that African American students feel toward other African American students were borne out in both the quantitative and qualitative data, but the qualitative data provide further contextualization of the findings by showing the key locations of interaction at this specific university, revealing the importance of specific context. The interviews also produced key findings pertaining to the manner in which African American students are perceived to represent themselves and the role of parental socialization. These findings will likely contribute to future research uncovering these dynamics in other settings, and possibly designing survey measures to study these processes across multiple settings.
There is not enough to space to continue detailing the many outstanding examples of mixed methods research being conducted by sociologists these days (e.g., adams & Trinitapoli, 2009; Andrews, 2001; Perry, 2009; Trinitapoli, 2007). However, I have attempted to share a sampling to demonstrate the creativity, the range in size, the variety in how integration of methods is achieved, and the ways in which these studies represent the key features of a pragmatic approach to research as outlined by Morgan (2007).
Concluding Thoughts
Sociologists continue to draw upon a wide range of data collection and analysis techniques, often combining these multiple methods to counterbalance each methods strengths and weaknesses and to develop rich layers of data to shed light on key sociological questions. Although not a particularly new approach, its popularity has resurged in recent years, therefore this reflection on from where we have come helps contextualize the relatively new term mixed methods research in the history of our discipline.
As we reflect on the path to the present, we see refinement over time in both the epistemology and the mechanics of qualitative and quantitative data collection. We observe the arc of tension in the discipline between the “quantitative” and “qualitative” paradigms. This tension appears to be weakening in recent years, curiously in parallel with the resurgence of interest in mixed methods research. It may be that the pragmatic paradigm and mixed method research have combined to provide a way through the tension—a methodological polytheism as Bordieu (2004) called it—promoting respect for all types of research methods when they are conducted rigorously. As more sociologists see the value in collecting both qualitative and quantitative data in a linked manner, we are seeing a further evolution of the “pragmatic” paradigm. This paradigm is based on the principles of intersubjectivity (interplay between objective and subjective stances), abduction (combining both inductive and deductive theory building), and transferability (inferring the reach of various findings) (Morgan 2007). Whether any given sociologist makes use of a mixed methods approach or not, increasingly we work to discover the merits of a wide variety of theoretical and methodological approaches and the discipline is better for it.
Of course, there are challenges to the maintenance of a pragmatic paradigm and mixed methods research in sociology. One challenge is that the majority of our undergraduate and graduate training programs are structured to reify the overly strict dichotomy and destructive distance between so called “qualitative” methods and “quantitative” methods. Graduate programs are assumed to specialize in one set of methods to the detriment of the others. General research methods classes keep the instruction of sets of methods separate and pose them as incompatible (usually indirectly). Pick up any sociology research methods text and you will quickly find the divisions. A better strategy would be to take the increased respect for and compatibility between various theoretical, methodological, and interpretive streams of research and structure our training programs on this (Onwuegbuzie & Leech, 2005). New courses and textbooks could focus on the holistic research process and teach the details, strengths, and weaknesses of each data collection method separately (not in some kind of typology that creates artificial and detrimental divides).
Another challenge posed by the pragmatic paradigm and/or mixed methods research is the competing efficiency of specializing in the use of one type of research method (Leahey, 2007a). Many argue it is asking too much of students and researchers to do the highest level of research using multiple, very different methods. This is a reasonable critique; however, in the age of increasing collaboration between researchers, especially across disciplines, the research team approach often facilitates a mixed methods research agenda. Further, there is still room to specialize within a mix of methods. In my own work, I almost always combine survey analyses with semi-structured interviews in a way that the survey suggests which respondents would be prime informants (Pearce 2002). In doing so, I rely on a certain toolbox I have developed, and continually seek the advice and feedback of expert colleagues. In other words, it is not necessarily easy, but then few research endeavors (whether mono- or multi-method) are.
Other concerns have arisen around the increase in mixed methods research within sociology. Leahy (2007b) expresses concern over the feasibility and ethics of mixed methods research that starts with survey analysis, claiming that recontacting subjects who have participated in a survey to do a semi-structured interview raises ethical issues. This is true, but can be addressed exactly how any longitudinal study addresses it: by having protocols that involve asking participants if they would be willing to be recontacted for future data collections and instituting procedures to always protect the privacy of participants. Further, Leahy (2007b) highlights how one must almost always be one of the original investigators of a survey data collection to have access to the identifying information that would make recontacting participants possible. In other words, for some types of mixed methods research, primary data collection is essential. That said, as data protections systems grow increasingly secure and data anonymizing tools become more sophisticated, the analysis of secondary data (both quantitative and qualitative) may become more feasible. Granted, there are certainly limitations to the analysis of qualitative data that are not collected by the analyst, so discussions should continue as to what our standards for the various types of data collection and analysis are (Small, 2009). Also, there are huge advantages to having mixed methods research projects that provide two forms of data from one set of participants (Pearce, 2002; Pearce & Denton, 2011). It allows for interesting comparisons and the transposing of data (e.g., quantitizing qualitative data and qualitizing quantitative data). Thus, there is much to be gained through this approach, however resource and time intensive it might be.
Another very reasonable concern in the growing combination of research methods is that one method (especially those that collect qualitative data) will be compromised in the service of another (usually a survey). Reality is that the majority of mixed methods research in sociology and most other fields has a design that privileges a survey research component in the project (e.g., qualitative data collected and analyzed to either help in the design of survey instruments or to help enrich the interpretation of findings from quantitative data). In fact, it is likely that those who were trained with more emphasis on survey data collection and/or analysis are more likely to take up mixed methods research than those who have previously specialized in methods involving the collection of qualitative data. Also, the advent of qualitative data analysis software and technology to more quickly transcribe has led to the possibility that larger masses of qualitative data can be collected in less time and with fewer resources. However, more is not always better. We thus risk applications of qualitative data collection that compromise the potential value added (Small, 2008, 2009).
Further attention to the above concerns is likely to greatly enhance the current state of research in sociology and further revive the pragmatic approach to a holistic research process to which any researcher can contribute. Contributions can be made through mono- or multi-method projects that all contribute towards the accumulation and interpretation of empirical evidence. The field of sociology has much to gain by continuing to embrace efforts to achieve a balance of induction and deduction, subjectivity and objectivity, contextual accuracy and generalization (Morgan, 2007). The competition and division between those that hold tightly to the stance that “qualitative” or “quantitative” methods are preferable to the other has been exposed as based more on politics than theory, or being more apparent than real (Hanson, 2008). Thus as we move forward it is important to acknowledge the long history of combining methods in sociology and draw on the numerous examples through time to continually improve our research designs and collaborations. The field will continue to benefit from a cooperative approach that values all methods and evaluates them in their own right.
Footnotes
Acknowledgements
Thank you to Burke Johnson, Kenneth T. Andrews, Michele Berger, Linda Burton, and Karolyn Tyson for insightful comments on earlier versions of this manuscript.
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
The author(s) received no financial support for the research, authorship, and/or publication of this article.