Mixed Methods Research Design for Pragmatic Psychoanalytic Studies

Abstract

Calls for more rigorous psychoanalytic studies have increased over the past decade. The field has been divided by those who assert that psychoanalysis is properly a hermeneutic endeavor and those who see it as a science. A comparable debate is found in research methodology, where qualitative and quantitative methods have often been seen as occupying orthogonal positions. Recently, Mixed Methods Research (MMR) has emerged as a viable “third community” of research, pursuing a pragmatic approach to research endeavors through integrating qualitative and quantitative procedures in a single study design. Mixed Methods Research designs and the terminology associated with this emerging approach are explained, after which the methodology is explored as a potential integrative approach to a psychoanalytic human science. Both qualitative and quantitative research methods are reviewed, as well as how they may be used in Mixed Methods Research to study complex human phenomena.

This paper proposes that the newly emerging science of Mixed Methods Research (MMR) offers a robust methodology for psychoanalytic research. MMR, also known as the “third community” of research in the social and behavioral sciences, seeks to capitalize on the strengths of quantitative and qualitative research traditions by combining these approaches into a complementary research design (Creswell 2009; Gelo, Braakmann, and Benetka 2008; Teddlie and Tashakkori 2009). However, recent recommendations from psychoanalytic scholars (Hauser 2006; Luyten, Blatt, and Corveleyn 2006; Wallerstein 2009) regarding the value of both quantitative and qualitative studies have not reviewed MMR as a specific research strategy for implementing their recommendations. In combining quantitative and qualitative strategies, MMR represents a paradigm shift from pure positivism or pure constructivism to a pragmatic research approach, advancing what have been viewed as disparate, at times antagonistic dichotomies to establish a complementary position. After reviewing recent discussions and controversies in psychoanalytic research, we provide an introduction to MMR rationale and design, providing a hypothetical research scenario of impasses in treatment to demonstrate how MMR design might be used. We next provide a brief review of basic qualitative and quantitative procedures as components of MMR design.

Approaches to Psychoanalytic Research

In recent years the field of psychoanalysis has taken more seriously the call for empirical research focusing on treatment process and outcome (Fonagy and Target 2002; Gerber et al. 2006; Hauser 2002, 2006; Kernberg 2006; Luyten, Blatt, and Corveleyn 2006). This interest stems in part from a growing commitment to demonstrating the validity of psychoanalytic constructs and the efficacy of psychoanalytic treatment, as both have drawn devastating critiques from academic and clinical science colleagues. Case reports, the primary source of clinical data over much of the field’s history, are judged by many analysts and scientists to provide insufficient evidence of treatment efficacy. Responding to these critiques, psychological science has become more statistically sophisticated; research studies now require larger samples, and data reduction using numerical techniques has evolved and come into wider use. Yet some analysts (e.g., Hoffman 2009), who in their practice treat complex problems involving various symptom constellations and character styles, object to reducing the rich individual narratives of case reports to the tabular data of numerical analysis.

Recently, echoing the position of Hauser (2002, 2006), Wallerstein (2009) has made the case for both qualitative and quantitative research in psychoanalytic studies. MMR procedures are designed to capitalize on the strengths of each method, aiming for both explanation and comprehension. Because MMR allows a rapprochement between empirical methods and hermeneutic traditions, in our opinion it best fulfills these calls for methodological pluralism in the psychoanalytic sciences.

Mixed Methods Research

At the beginning of any research endeavor, the researcher selects a topic and then methods appropriate to the line of inquiry. Gelo, Braakmann, and Benetka (2008) contrast quantitative methodology aimed at explanation by establishing observable and measurable facts, what we observe, with qualitative procedures aimed at comprehension, defined as “the reconstruction of how someone else has established connections between facts through the regularities they observe” (p. 271; emphasis added). A monomethod design will suffice if the researcher determines that all questions/hypotheses may be adequately addressed by either quantitative or qualitative methods alone. In some studies, however, neither method alone is sufficient to provide a comprehensive understanding of the phenomena to be investigated. In such complex situations it can be useful to plan, at the outset, a comprehensive research strategy using MMR methodology.

In 1959 Campbell and Fiske published their landmark paper recommending that researchers use “multitrait-multimethod” procedures to enhance the reliability and validity of one’s findings. These authors were advocating for multiple quantitative techniques in research. Operating from a similar premise, MMR seeks to integrate both qualitative and quantitative methods into research design to develop a comprehensive understanding of complex human phenomena (see Table 1). In the MMR paradigm, qualitative (QUAL) and quantitative (QUAN) research are seen not as dichotomous but as existing on an “interactive continuum” (Newman and Benz 1998, cited in Gelo, Braakmann, and Benetka 2008), with one method potentially broadening or deepening our understanding of a particular research question. MMR practitioners consider their approach to have its roots in pragmatism (Creswell 2009; Tashakkori and Teddlie 2003; Teddlie and Tashakkori 2009). Pragmatism as a school of thought originates in the work of C. S. Peirce, William James, and John Dewey and is today expounded in the work of Richard Rorty and other postmodern thinkers. Primarily, pragmatism is committed not to any one system of philosophy but rather to understanding the problem at hand with all available methods. Creswell (2009) describes pragmatically driven research as informed by, or interacting with, social, historical, political, and other specific contexts. Researchers are free to select the method of study that best suits the research question; hence, a plurality of methods, tolerance for different worldviews, and novel strategies of inquiry and research design characterize such studies.

Table 1.

Qualitative, Mixed Methods, and Quantitative attributes

Parameter	Qualitative	Mixed Methods	Quantitative
Paradigm	Constructivism	Pragmatism	Positivism/Post-Positivism
Descriptive Approach	Idiographic—individualizing	Appropriate number for quantitative analysis, appropriate number for qualitative methods	Nomothetic—generalizing
Sampling	Purposive—N = 3-20	Both purposive and probability sampling	Probability—Based on Power and Effect Size determination
Goal of Analysis	Comprehension —interpretation —contextualization	Comprehension and Explanation	Explanation —prediction —generalization
Data Source	Narrative —interview data —writing samples —focus groups	Narrative and numeric	Numeric/Tabular —questionnaires —surveys —Q-sort —psychometric instruments
Type of Data Analysis	Thematic—units of meaning Inductive Reasoning	Integration of thematic and statistical data	Statistical Analysis —descriptive —inferential Deductive Reasoning
Validity	Descriptive Interpretive Explanatory	Combined strength of validity from qualitative strand and reliability from quantitative strand	Internal Validity Generalizability Statistical—Conclusion Validity Statistical Construct Validity

Table constructed from Teddlie and Tashakkori (2009) and Gelo, Braakmann, and Benetka (2008).

In a recent review of research articles published in three major psychoanalytic journals (Tillman, Oberwager, and Agar 2010), we identified studies using quantitative or qualitative research methods, and some using both, though none of these explicitly claimed MMR methodology. Several studies were noted to make use of quantitative data analysis, with a qualitative interview/video analysis included in the research report. Craige (2002) studied posttermination mourning in candidates using a questionnaire returned by 121 candidates who had terminated their personal analysis; a semistructured interview was then conducted with 20 participants selected from representative groups based on data analysis of responses to the questionnaire. In another study of posttermination improvement, Falkenström et al. (2007) used questionnaires followed by multiple case study interviews to learn more about the experience of change. They used interview data to look “at each patient in relation to him- or herself in order to highlight the individual post-treatment process of each patient in the context of all available data on that case” (p. 636). Clearly using a mixed methods strategy, this study yielded information about treatment process and outcome. Studying variables associated with leadership processes, Rudden, Twemlow, and Ackerman (2008) conducted a study using systematic qualitative analysis of video recordings of group meetings, along with a quantitative rating scale to test hypotheses framed at the study’s outset. Each of these studies was designed to employ quantitative and qualitative procedures together, the hallmark of MMR.

In the other studies we reviewed that used both quantitative and qualitative methods, we observed that researchers tend to start with a quantitative project and then turn to anecdotal evidence of a qualitative sort to narrate their findings. By our reading, the qualitative piece was often a post hoc procedure that did not conform to a carefully designed qualitative method of analysis and interpretation. Other researchers use potentially qualitative interview data transformatively, by quantifying interviews using various rating scales.

Indicative of the growing sophistication of MMR, Tashakkori and Teddlie (2003) were able to identify over forty types of MMR designs and procedures. More recently these methods have been summarized and streamlined to better describe the practice of MMR today. There are now four major mixed methods designs, with variation in each category: (1) triangulation design; (2) embedded design; (3) explanatory design; (4) exploratory design. Each of these four strategies may be conducted concurrently or sequentially. Finally, the qualitative and quantitative strands of research may be equal in weight, or one may be the dominant method (Creswell and Plano Clark 2007; Creswell 2009; Gelo, Braakmann, and Benetka 2008; Teddlie and Tashakkori 2009). We will now review some basic MMR research design strategies.

Concurrent or Sequential Design (1-Phase or 2-Phase Project)

Once the researcher decides to use both quantitative and qualitative methods, the next question is whether data collection will be done concurrently (simultaneously) or sequentially. With concurrent data collection, both quantitative and qualitative aspects of the study are conducted at the same time. In the concurrent method, the two sets of data are analyzed and then merged using either transformation or comparison. In the first, the researcher transforms qualitative meaning units into numeric units and works with the data both quantitatively and qualitatively. Comparison of the separate data analyses may be effected through the use of a matrix in which quantitative data are linked to qualitative units, producing convergent validity.

Sequential designs are two-step procedures. One methodological strand of the research is completed and analyzed, after which the other method is employed in the second step. For example, a quantitative study may be carried out, after which a sample of participants identified from the quantitative analysis are interviewed with the aim of contextualizing these data using qualitative methods. This is especially useful for understanding extreme cases and outliers, since the quantitative strategy identifies the central tendency of the sample, and information about the meanings and structure of outliers can be overlooked. Limitations of sequential designs include the greater amount of time needed to conduct the study and the subject attrition occasioned by this longer duration.

Weighting: Qualitative, Quantitative, or Balanced

After choosing a particular qualitative or quantitative approach, the researcher must then decide whether qualitative and quantitative approaches will be equally weighted or whether one will dominate the study, with the other playing a secondary role. For notational purposes, QUAN and quan, or QUAL and qual, signify the priority given to each method, with all-capital abbreviations indicating the dominant paradigm and italic lowercase letters indicating the secondary method. The priority for weighting is dependent on the aims of the researcher, what the researcher hopes to emphasize for the intended research audience, and whether the research is primarily inductive (emerging themes in qualitative procedures) or deductive (hypothesis testing in quantitative procedures) (Creswell 2009).

Triangulation, Embedded, Exploratory, Explanatory

The most popular method used in MMR is triangulation, in which different data sources or methods are used to study the same topic (Creswell and Plano Clark 2007). These data are collected concurrently, the methods given equal weight and then used to identify patterns of convergence or divergence. For example, interview data may be compared to observational data, a Rorschach and a Thematic Apperception Test (TAT) may be used to study personality, or qualitative and quantitative methods may be used to study a single variable. Often it is assumed that the weaknesses in one test or method will be counterbalanced by the strength of the other method, increasing claims of validity. The researcher expects that the weaknesses of one method do not overlap the weaknesses of the other, and that the strengths of the methods are additive. Mays and Pope (2000) caution about this leap to validity based on the strength/weakness balance and observe that triangulation may be a means to ensure the comprehensiveness of a study rather than to validate its results.

A design is said to be embedded when one method is primary and the second is subordinate, playing a supporting role for the first. Either a concurrent or a sequential approach may be used with an embedded design. In a concurrent embedded design both qualitative and quantitative data are simultaneously collected, but one method is dominant and guides the study (Creswell 2009). The secondary method is said to be embedded or nested in the primary method. For example, in a primarily quantitative study the data analysis may address the outcome while the embedded qualitative procedure explains the processes of the participants in the study. A theoretical example of such a study would include a quantitative measure of patient outcome via patient assessment and then a selection of therapists of these patients to be interviewed about their understanding of the treatment process. Mixing the data would give information of outcome from two perspectives using two methods. Two types of sequential embedded designs will be reviewed next.

Explanatory and exploratory designs are always sequential in nature, and weighted so that either the quantitative or the qualitative method dominates. Because they are nested, they may both be considered forms of embedded designs. The primary difference between an embedded design and exploratory and explanatory designs is that unlike the latter, an embedded design may be either concurrent or sequential.

In explanatory designs the QUAN arm is the primary data type and qual findings secondary. For example, a large QUAN study might be conducted, the data analyzed, and outliers identified. Interviews would then be conducted with outliers, as well as with participants clustered at the mean, in order to have a better qual explanation for potentially different phenomenological features not captured by QUAN measures. Quantitative statistical procedures make use of effect sizes to identify clinically significant differences, while the addition of qualitative procedures provides a narrative understanding of the lived effects of the group difference, and may deepen our understanding of various pathways to particular outcomes that can be used to refine future quantitative efforts. Using effect sizes as a starting place followed by qualitative data collection/analysis illuminates the subjective meanings of these differences. When expected research findings or treatment interventions are not effective, as measured by quantitative measures, a qualitative component may generate hypotheses about why individuals do not respond as predicted.

Exploratory designs are primarily QUAL, with the quan arm secondary. Such a design is most often used to first describe a phenomenon via interviews. From the information gained, a quantitative instrument may be constructed to measure the phenomenon. Test construction and development often begin with a qualitative assessment of the topic to understand the basic structure of the phenomena prior to attempts at measurement. In this procedure, qualitative data are used to produce items for future quantitative measures, increasing the likelihood that such measures will demonstrate construct validity.

Data Analysis and Integration

In mixed methods data analysis, the qualitative and quantitative aspects of the study must not only be reported, but also integrated or linked together to draw conclusions about the topic under examination. One might think of this as a dialectical procedure in which the QUAN↔QUAL data conversation acts as a site of tension through which the researcher arrives at an integrated synthesis. Geertz (1983) has used the concept of “continuous dialectical tacking” to describe data analysis moving between “the most local of local detail to the most global of global structure in such a way as to bring them into simultaneous view” (p. 69). This analogy applies to MMR, where qualitative data may be considered “local,” with quantitative data providing “global structure” (Greene 2007).

Successful data integration should produce findings that are greater than the sum of the parts of either the quantitative or qualitative analysis alone (Woolley 2009). Bazeley (2009) summarizes the multitude of ways data integration through analysis may be done, including (1) “Employment of the results from analysis of one form of data in approaching the analysis of another form of data”; (2) “Comparison of coded or thematic qualitative data across groups defined by categorical or scaled variables”; (3) “Pattern analysis using matrices”; (4) “Conversion of quantitative data into narrative form in the service of profiling”; (5) “Conversion of qualitative to quantitative coding to allow for descriptive, inferential or exploratory statistical analysis”; (6) “Extreme and negative case analysis”; (7) “Inherently mixed data analysis, where a single source gives rise to both qualitative and quantitative information”; and (8) “Intensive case analysis” (p. 205). Teddlie and Tashakkori (2009) describe the process of data integration as allowing the “two sets of analyses to ‘talk to each other’ in at least a semi-iterative manner” (p. 266), noting that such a conversation may lead to both convergent and divergent results.

Designing an MMR Study: A Hypothetical Study of Treatment Impasses

Impasses and ruptures in psychodynamic treatment have been discussed in clinical case reports (Aron 2006; Ehrenberg 2000; Elkind 1992; Tillman 1999) and have been studied using various measures of alliance (Ackerman, Hilsenroth, and Knowles 2005; Horvath and Greenberg 1989; Safran et al. 2005). Most case reports of impasse or rupture are written from therapists’ or researchers’ points of view, based on what they observe in clinical work or through various measures of alliance or session quality associated with premature endings (Eubanks-Carter, Muran, and Safran 2010). These case reports and quantitative studies often do not include sufficient data directly from patients and what they know about their experience of impasses. Recognizing that impasses/ruptures are complex and poorly defined phenomena, our hypothetical researchers plan to study impasse/rupture from the patient’s perspective to understand more about unsatisfactory termination.

In the imaginary study the researchers want to test several hypotheses about impasses using quantitative measures, to verify whether therapists’ descriptions of patient characteristics contributing to an impasse are accurate. Are patients involved in impasses less resilient and unable to manage disappointment, misrecognition, or failures? Are they significantly different from the group reporting satisfactory terminations? The researchers also want to capture patients’ lived experience via phenomenological interviews in order to understand from their perspective how they see themselves as contributing to the impasse and what they see as the therapist’s contribution.

The researchers determine that they will use an MMR design that will allow them to test hypotheses using quantitative procedures, with a qualitative arm consisting of phenomenological interviews. Reflecting on their resources, research aim, and hypotheses, the researchers decide on a concurrent triangulation design, combining patient questionnaires with semistructured interviews asking about their experience of a treatment impasse (QUAN + QUAL). Data will be collected in a clinic setting where demographic information is available for all patients, allowing the researchers access to data about diagnosis, gender, length of time in treatment, age, and other salient descriptive variables. The researchers also have access to patient satisfaction follow-up data identifying patients who report a satisfactory termination process, as well as those reporting an impasse/rupture leading to termination.

For the QUAN portion of the study two hypotheses will be tested: patients in the impasse group will (1) show significantly lower scores on a self-report measure of treatment alliance and (2) show significantly lower scores on a self-report measure of resilience. The researchers select self-report measures with demonstrated psychometric reliability and validity. The planned data analysis includes the use of inferential statistics; to determine the minimal sample size needed to detect statistically significant differences, the researchers set their power using Cohen’s convention of .80 and determine that they will need a sample of 90. They plan to recruit 45 participants from the impasse group and 45 from the satisfactory outcome group to complete self-report measures. The researchers anticipate an effect size of Cohen’s d = .50, for between-group differences in measures of resilience and alliance.

For the QUAL portion of the study, the researchers interview a sample of participants reporting a treatment ending in an impasse/rupture using the bracketing question “Tell me about your experience of being very stuck in your psychotherapy and what you think contributed to being so stuck. We are interested in what you think you may have contributed to being stuck, what your therapist may have contributed, and anything else that you think may have been a factor.” The interviews will be audio-recorded, and typed transcripts will be used to analyze the narrative data using Interpretative Phenomenological Analysis (Smith, Flowers, and Larkin 2009). Every other participant in the therapist-identified impasse group will be asked to complete the interview portion of the study, with the goal of interviewing twelve participants. The assumption is that not every eligible participant will agree to participate in the interviews. After designing the study and receiving approval from their Institutional Review Board to use human subjects, the researchers are ready to recruit their subjects and begin collecting data.

After collecting data, self-report measures are scored and data analysis proceeds, using appropriate inferential statistics techniques. Using the clearly defined procedure of Interpretative Phenomenological Analysis for analyzing qualitative data, the researchers identify a coherent set of six themes associated with the experience of impasse. The quantitative analysis confirms the hypothesis that participants reporting a therapeutic impasse score significantly lower on the alliance measure than do those in the satisfactory outcome group, and the effect size is also robust. They are surprised by the finding that the impasse group does not score significantly lower on the measure of resilience than do participants in the satisfactory outcome group, and hope that the interviews will shed some light on this finding.

The researchers discover that even though impasses are two-person events, hypotheses in the quantitative arm of the study may have overpathologized participants in the impasse/rupture group, leading them to wonder if this is an artifact of the literature, which produces primarily therapists’ descriptions of impasses and ruptures. The researchers analyze the interview data to see what participants actually say about their experience of the impasse, hoping this might increase their understanding of their hypotheses and locate the phenomena more fully within the dyad. They look at the categories they have identified and develop typologies of impasse that show a more complicated picture than the concepts of resilience and alliance. In addition, they use a contextualizing strategy that situates the impasse in transference dynamics contributing to the frustrating repetition of scenes from the past, including an hypothesized dimension of role-responsiveness in the therapist contributing to and sustaining the demise of the treatment.

Once the quantitative and qualitative analyses are complete, the researchers agree to a parallel mixed data analysis (Teddlie and Tashakkori 2009, p. 266), which allows for fidelity to the separate methods of data analysis. Inferences are then made from each strand of the research, and the investigators use data from each analysis to form meta-inferences, the integrative product of the QUAN and QUAL portions of the study. The parallel data analysis procedure may lead to convergent or divergent meta-inferences. Through triangulation of the quantitative and qualitative data, a more complete and complex picture of the conditions of impasse from the patient’s perspective becomes evident.

In our hypothetical study our investigators are presented with the conundrum of a quantitative hypothesis about resilience that is not confirmed, and surmise that resilience as a trait measure is not an adequate measure of a state-related, transference-driven event such as impasse. From the interviews our researchers learn that many of the participants in the impasse condition are actually quite resilient, that they do not submit to the misrecognition so characteristic of an impasse, and that what may appear on the surface to be obstinacy is really a subtype of resilience that may not indicate resistance to a working alliance or reluctance about joining a misalliance within the transference/countertransference matrix. This hypothetical research project provides just one example of how researchers might study complex psychodynamic phenomena using MMR procedures.

The researchers are now in a position to design a second phase of the study, using what they have learned from the interviews to construct a different questionnaire, one that takes into account the dynamics of repetition, the concept of “wounding in areas of primary vulnerability” (Elkind 1992), and the interactional field of the dyad. Additionally, they decide in their next study to measure therapist/patient pairs, piloting their new quantitative measure of the features specific to impasses as reported by patients and earlier by therapists.

Qualitative and Quantitative Procedures

As noted above, MMR design is selected when neither quantitative nor qualitative methods alone are sufficient for answering research questions involving complex dynamics. MMR is a planned research approach, not simply a tacking of anecdotal narration onto quantitative findings. To conduct MMR studies, researchers need basic competence in both qualitative and quantitative methods. In this section we review the most basic considerations involved in both.

Qualitative Methods

Qualitative methods and narrative research involve a range of well-established procedures for data collection and analysis. For qualitative studies to be methodologically sound and reliable, clarity and rigor are important aspects of research design. Researchers may opt for structured, semistructured, or open-ended interviews. Other sources of qualitative data include videotapes, audiotapes, or other transcribed narrative documents. Once the format of data collection is determined, researchers need to think about their approach to the study, and what method will be sufficient to achieve their goals. Detailed single-case study is one approach when the aim is discovery and theory building (Grünbaum 1984; Wallerstein 2009). Tested interview approaches include phenomenological interviews and analysis (Giorgi 2005, 2008; Kvale 1996; Smith, Flowers, and Larkin 2009; Smith and Osborn 2003; von Eckartsberg 1986) and multiple case research (Mishler 1986; Rosenwald, 1988). The Psychoanalytic Research Interview (Cartwright 2004) and the Listening Guide (Gilligan et al. 2003) are other interview and analysis procedures a researcher might use in the qualitative portion of a study.

The most important point is that the researcher identifies and employs a clearly defined method in order to promote the reliability and validity of the procedure. Statements such as “a qualitative analysis was used,” without careful elaboration of the method employed, harm the field of qualitative studies. Reliability of qualitative methods requires that another researcher be able to follow the procedure to repeat the study using accurately replicated steps. In MMR, attention to the qualitative arm from the beginning of the study will ensure that carefully planned procedures are in place and make the contribution of qualitative data more robust than that of studies presenting qualitative data as if it were an afterthought.

Quantitative Methods

Quantitative approaches excel in their ability to generalize findings across populations and to predict outcomes. Quantitative methods typically (though not exclusively) refer to the application of statistical analyses to numerical data collected in an effort to test an a priori hypothesis. Although quantitative methods, especially RCTs, are often viewed as the “gold standard” of research, these methods too are dependent on careful consideration of reliability and validity. As in qualitative procedures, achieving adequate levels of reliability in data collection and coding process is critical for ensuring the validity of a study, as is the use of standardized benchmarks in assessing reliability (see, e.g., Landis and Koch 1977).

Whether the quantitative approach involves a large N or is an N = 1 study, it is particularly important that issues related to proper sampling size be addressed. Sampling should be conducted in such a manner that the researcher can address how the study’s findings will or will not generalize across groups. In addition, using measures that reduce the effects of confounds (or including measures allowing the researcher to account for possible confounds) and relying on well-defined, carefully constructed operational definitions of the constructs under investigation add significantly to the validity of findings.

Finally, in designing research, conducting a power analysis ensures that the sample size of the study will be sufficient for detecting statistically significant differences using inferential statistics. But because statistically significant differences refer simply to the likelihood that a Type II error has not occurred, the next step is to determine the effect size (Cohen’s d, Glass’s delta, Hedge’s g, etc). The effect size standardizes the magnitude of the measured differences. Simply put, statistical significance demonstrates the precision of the estimate, while effect size tells us about the practical significance of the magnitude of that difference (Ellis 2010).

Discussion

Psychoanalytic researchers are uniquely poised to make rich use of MMR. Our clinical skills train us to listen to the deep structure of life narratives, constructing units of meaning and interpreting the multilayered complexities of a life story. Using skills developed in clinical training for sophisticated qualitative research endeavors may be a particular strength of our discipline and promote new discoveries. Likewise, behavioral scientists or those trained in medical/natural science research are in a position to operationalize complex phenomena, aggregating data and explaining various processes and outcomes via RCTs, time series analysis, single-case experimental design, metaanalysis or other quantitative methods, producing robust confirmation or refutation of hypotheses. For some research questions, combining these traditions and procedures using MMR may produce complex understandings of human behavior and experience that quantitative or qualitative procedures alone may miss.

MMR has several limitations as well. It is a resource- and time-intensive procedure, and for that reason when a monomethod design will suffice, MMR may not be the most efficient procedure. The wisdom of Occam’s razor, the law of parsimony, reminds us that what can be done with fewer means is done in vain with many; and this holds true for research design. Another limitation of this relatively new methodology is that procedures for mixing or integrating the strands of a project are still being developed and require patience and creativity on the part of the research team. The MMR scholars Johnson and Onwuegbuzie (2004) review the long history of dispute between the traditions of quantitative and qualitative research, noting that some regard the methods as orthogonal to each other and believe they should remain so. Methodological research purity may be desirable in some disciplines, but we think psychoanalytic studies will benefit from the integration of data from both qualitative and quantitative perspectives. Such integration will enhance research endeavors, and may allow for some rapprochement across the clinician/researcher divide. Another possible limitation is that MMR is associated with the social sciences but not the medical sciences, and so may be viewed in the latter as having lower status. Yet our debates about the psychoanalytic endeavor suggest we are indeed a hybrid of medical science and hermeneutic practices, so MMR may actually allow the two traditions to work together rather than in opposition.

Psychologists are familiar with a long-standing tradition of MMR in psychological testing, in which a battery of tests often includes quantitative approaches (Wechsler Adult Intelligence Scale, Rorschach Comprehensive System) and hermeneutic approaches to thematic analysis of the TAT or content analysis of the Rorschach. Writing a psychological test report for an individual is a microcosm of MMR, using quantitative and qualitative instruments in an integrated fashion, with the aim of describing the unique characteristics of the individual while also locating the person within group norms from psychometric studies.

MMR develops a dialectical tension between quantitative and qualitative approaches in the behavioral and human sciences. Geertz’s concept of “continuous dialectical tacking” provides the opportunity for a thick description of the subject and the object of study at the level of both local detail and global structure. Such increasing diversity may lead some to worry that research approaches will be based on a dilution of quantitative or qualitative techniques rather than on a distillation and integration of the best elements of each. This debate generally replicates the tension in the move from modern to postmodern thinking, a move whereby diversity and pluralism are now thought to enhance scholarship rather than diminish academic rigor, though detractors of postmodernism associate it with casual relativism.

In MMR we advocate for fidelity to both the quantitative and the qualitative aspects of the study, that is, using qualitative data to understand the deeper structure of the experience of a particular event or subjective reflection and quantitative procedures to capture group norms and differences. In our view, MMR is not a collapsing of distinct traditions into an undifferentiated procedure, but rather a thoughtful mixing of distinctive procedures in a sophisticated conversation. Teddlie and Tashakkori (2009) use the phrase “let the data talk to each other,” and, we would add, talk to each other in the mother tongue of each tradition, working across the boundary to deepen understanding, not simply erasing the boundary.

This may be a difficult transition for the individual researcher, often trained early in his or her career to pledge allegiance to one method or the other and dismissive of methods in which they have not been trained. Working in a research group, with colleagues who have declared allegiance to their training yet are interested in the conversation across the boundary, may be one way of moving toward a truly MMR spirit of inquiry. Interdisciplinary collaboration will likely strengthen our methodological rigor and the conclusions drawn from psychoanalytic studies that must accommodate the complexity of human cognition, affect, and behavior.

Footnotes

The authors thank their 2009 summer research interns Kayla Agar, Benjamin Johnson, Jordan Nejaime, Katherine Oberwager, and Carrie Olsen.

References

Ackerman

Hilsenroth

Knowles

(2005). Ratings of therapist dynamic activities and alliance early and late in psychotherapy. Psychotherapy: Theory, Research, Practice, Training 42:225–231.

Aron

(2006). Analytic impasse and the third: Clinical implications of intersubjectivity theory. International Journal of Psychoanalysis 87:349–368.

Bazeley

(2009). Editorial: Integrating data analyses in mixed methods research. Journal of Mixed Methods Research 3:203–207.

Campbell

Fiske

(1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin 56:81–105.

Cartwright

(2004). The psychoanalytic research interview: Preliminary suggestions. Journal of the American Psychoanalytic Association 52:209–242.

Craige

(2002). Mourning analysis: The post-termination phase. Journal of the American Psychoanalytic Association 50:507–550.

Creswell

J.W.

(2009). Research Design: Qualitative, Quantitative, and Mixed Methods Approaches. 3rd ed. Thousand Oaks, CA: Sage Publications.

Creswell

J.W.

Plano Clark

V.L.

(2007). Designing and Conducting Mixed Methods Research. Thousand Oaks, CA: Sage Publications.

Ehrenberg

D.B.

(2000). Potential impasse as analytic opportunity: Interactive considerations. Contemporary Psychoanalysis 36:573–586.

10.

Elkind

S.N.

(1992). Resolving Impasses in Therapeutic Relationships. New York: Guilford Press.

11.

Ellis

P.D.

(2010). The Essential Guide to Effect Sizes: Statistical Power, Meta-analysis, and the Interpretation of Research Results. Cambridge: Cambridge University Press.

12.

Eubanks-Carter

Muran

J.C.

Safran

J.D.

(2010). Alliance ruptures and resolution. In The Therapeutic Alliance: An Evidence-Based Guide to Practice, ed. Muran

Barber

J.P.

New York: Guilford Press, pp. 74–94.

13.

Falkenström

Grant

Broberg

Sandell

(2007). Self-analysis and post-termination improvement after psychoanalysis and long-term psychotherapy. Journal of the American Psychoanalytic Association 55:629–674.

14.

Fonagy

Target

(2002). The history and current status of outcome research at the Anna Freud Centre. Psychoanalytic Study of the Child 57:27–60.

15.

Geertz

(1983). Local Knowledge: Further Essays in Interpretive Anthropology. New York: Basic Books.

16.

Gelo

Braakmann

Benetka

(2008). Quantitative and qualitative research: Beyond the debate. Integrative Psychological & Behavioral Science 42:266–290.

17.

Gerber

A.J.

Kocsis

J.H.

Milrod

Roose

S.P.

(2006). Assessing the quality of randomized controlled trials of psychodynamic psychotherapy. Journal of the American Psychoanalytic Association 54:1307–1313.

18.

Gilligan

Spencer

Weinberg

M.K.

Bertsch

(2003). On the Listening Guide: A voice-centered relational method. In Qualitative Research in Psychology: Expanding Perspectives in Methodology and Design, ed. Camic

Rhodes

Yardley

Washington, DC: American Psychological Association, pp. 157–172.

19.

Giorgi

(2005). The phenomenological movement and research in the human sciences. Nursing Science Quarterly 18:75–82.

20.

Giorgi

(2008). Concerning a serious misunderstanding of the essence of the phenomenological method in psychology. Journal of Phenomenological Psychology 39:33–58.

21.

Greene

J.C.

(2007). Mixed Methods in Social Inquiry. San Francisco: Wiley.

22.

Grünbaum

(1984). The Foundations of Psychoanalysis: A Philosophical Critique. Berkeley: University of California Press.

23.

Hauser

S.T.

(2002). The future of psychoanalytic research: Turning points and new opportunities. Journal of the American Psychoanalytic Association 50:395–405.

24.

Hauser

S.T.

(2006). Studying psychoanalysis: Asking and answering the hard questions. Journal of the American Psychoanalytic Association 54:561–569.

25.

Hoffman

I.Z.

(2009). Doublethinking our way to “scientific” legitimacy. Journal of the American Psychoanalytic Association 57:1043–1069.

26.

Horvath

Greenberg

(1989). Development and validation of the Working Alliance Inventory. Journal of Counseling Psychology 36:223–233.

27.

Johnson

R.B.

Onwuegbuzie

A.J.

(2004). Mixed methods research: A research paradigm whose time has come. Educational Researcher 33:14–26.

28.

Kernberg

O.F.

(2006). The pressing need to increase research in and on psychoanalysis. International Journal of Psychoanalysis 87:919–926.

29.

Kvale

(1996). InterViews: An Introduction to Qualitative Research Interviewing. Thousand Oaks, CA: Sage Publications.

30.

Landis

J.R.

Koch

G.G.

(1977). The measurement of observer agreement for categorical data. Biometrics 31:159–174.

31.

Luyten

Blatt

S.J.

Corveleyn

(2006). Minding the gap between positivism and hermeneutics in psychoanalytic research. Journal of the American Psychoanalytic Association 54:571–610.

32.

Mays

Pope

(2000). Qualitative research in healthcare: Assessing quality in qualitative research. British Medical Journal 320:50–52.

33.

Mishler

E.G.

(1986). Research Interviewing. Cambridge: Harvard University Press.

34.

Newman

Benz

C.R.

(1998). Qualitative-Quantitative Research Methodology: Exploring the Interactive Continuum. Carbondale: Southern Illinois University Press.

35.

Rosenwald

(1988). A theory of multiple-case research. Journal of Personality 56:239–264.

36.

Rudden

Twemlow

Ackerman

(2008). Leadership and regressive group processes: A pilot study. International Journal of Psychoanalysis 89:993–1010.

37.

Safran

Muran

Samstag

Winston

(2005). Evaluating alliance-focused intervention for potential treatment failures: A feasibility study and descriptive analysis. Psychotherapy: Theory, Research, Practice, Training 42:512–531.

38.

Smith

J.A.

Flowers

Larkin

(2009). Interpretative Phenomenological Analysis: Theory, Method and Research. Thousand Oaks, CA: Sage Publications.

39.

Smith

J.A.

Osborn

(2003). Interpretative phenomenological analysis. In Qualitative Psychology: A Practical Guide to Research Methods. Thousand Oaks, CA: Sage Publications, pp. 51–80.

40.

Tashakkori

Teddlie

(2003). Handbook of Mixed Methods in Social and Behavioral Research. Thousand Oaks, CA: Sage Publications.

41.

Teddlie

Tashakkori

(2009). Foundations of Mixed Methods Research. Thousand Oaks, CA: Sage Publications.

42.

Tillman

J.G.

(1999). Psychosis as the sickness unto death: Treatment impasse, consultation and resolution. American Journal of Psychoanalysis 59:119–125.

43.

Tillman

J.G.

Oberwager

Agar

(2010). Research publications in three psychoanalytic journals, 2001–2008. Poster presented at the American Psychoanalytic Association poster session, New York, January 15.

44.

von Eckartsberg

(1986). Life-World Experience: Existential-Phenomenological Research Approaches in Psychology. Washington, DC: Center for Advanced Research in Phenomenology / University Press of America.

45.

Wallerstein

(2009). What kind of research in psychoanalytic science? International Journal of Psychoanalysis 90:109–133.

46.

Woolley

C.M.

(2009). Meeting the mixed methods challenge of integration in a sociological study of structure and agency. Journal of Mixed Methods Research 3:7–25.