Mixed Method Lessons Learned From 80 Years of Q Methodology

Overview

Although Q, with its qualitative–quantitative mix, began in 1935, Creswell (2010) estimates that the mixed method research movement began around 1988. The number of articles, books, and journals dedicated to using mixed methods has been steadily increasing since that time (Tashakkori & Newman, 2010). Similarly, Hesse-Biber (2010) identified mixed methods research as both old and emergent. Historically, as she notes, researchers in the 1800s and 1900s implemented mixed methods research without differentiation or note. Yet, since its inception by William Stephenson in his 1935 publication in Nature, the interplay between qualitative and quantitative within Q methodology has been typically misunderstood outside, and sometimes even inside, the Q community.

Stephenson (1986) noted the possible confusion within his methodology’s mixing of laws, theory, instrumentation, and application in ways contrary to popular and sometimes dogmatic objective–positivist stands. It seems that this “confusion” existed among the majority of behavior and social scientists during Stephenson’s career. Fifteen years after his death, the term qualiquantology, developed specifically to describe Q by Stenner and Stainton-Rogers (2004), was meant to draw greater attention to this qualitative–quantitative mixture throughout Q’s methodology. Specifically, Stenner (2011) later reaffirmed that he and Stainton-Rogers created this rather cumbersome word to represent “the peculiarly hybrid qualities of Q methodology” (p. 192). He reiterated the warning he and Stainton-Rogers (Stenner & Stainton-Rogers, 2004) had provided earlier which was that simply combining a qualitative dimension to a quantitative dimension . . . or vice versa was not sufficient to describe Q methodology. Instead, at least when combining qualitative and quantitative in Q, Stenner wanted to make clear that the combination was a discomforting hybrid.

Hybridity ought to be discomforting, since any genuine hybrid represents a significant reformation in the bodies that are brought together in forming it. Hybridity pierces the boundaries of identity and opens up the difference of otherness. By contrast, merely adding a qualitative dimension to a quantitative study or vice versa does not constitute hybridity and may be far from discomforting. (Stenner & Stainton-Rogers, 2004, p. 101)

Stenner (2011) further clarifies this idea by explaining in some disciplines, such as psychology, qualitative may be defined as the absence of quantitative, meaning numerical data. This becomes more complicated in psychology where subjective variables such as intelligence, attitude, and self-esteem are not directly observable and/or measurable. Stenner continues that identifying these unobservable variables was the purpose of inventions such as factor analysis which allowed the identification of these subjective variables statistically. The reduction into factors presumes that these factors are real but unobservable latent variables and was a response to the lack of genuinely quantitative methods in psychology.

A purpose, therefore, of such methods as R is to reduce and/or eliminate the qualitative and subjective. Instead, Q methodology maximizes these and makes it the main focus (Stenner, 2011). Although Q utilizes factor analysis, the data are different than that of R methodology. The Q sort data represent expressions of qualitative intensity related to feeling or value (Stenner, 2011; Stephenson, 1953b). This is in contrast with Likert-style questionnaires, attitude scales, personality measures, and other familiar quasi-quantitative psychological procedures which attempt to provide objective variables which theoretically vary only in quantitative, not qualitative, terms (Stenner, 2011). Stenner (2011) summarizes his description of Q and its comparison with R as follows:

Q operates with an ontology in which the ultimate realities are neither subjects nor objects, but actual occasions of experience. . . . This is no ordinary “mixing” of methods and it is precisely not a matter of an objective “natural world” being contrasted with a socially constructed and subjective “human world”: it is a qualiquantology. (p. 201, italics in original)

Thus, Stenner (2011) has made qualiquantology distinct from mixed methods because of the importance of considering the methodology, philosophy, epistemology, and ontology within Q. This is where the discomfort lies. This is not unlike the discussion by Creswell (2010) about the development of the landscape of mixed methods. Debates about mixing paradigms often lead to questions about whether or not this is possible; if possible, are restrictions required for that mixing? Certainly, discomfort is noticeable within such topics of mixing paradigms within the mixed methods community.

It would seem that this is why Stenner and Stainton-Rogers (2004) felt the need to create their own term, qualiquantology, to represent the uncomfortable, hybridity that exists with Q. In other words, to them, the phrase “mixed method” does not represent that kind of hybridity. The discomfort relative to this “mix” is quite evident when considering the various controversies and misunderstandings during Q’s history (Brown, 1980). Difficulties during Stephenson’s career are noted by various researchers including Good (2010). Good (2010) notes difficulties such as Stephenson’s struggles to secure a tenured position until he reached the University of Missouri in 1958 (23 years after introducing Q). There, Stephenson, with PhDs in physics and psychology, was in the School of Journalism teaching research courses related to theories of mass communication.

Generally, there are misconceptions about the overarching methodology of Q. These misconceptions are often connected to addressing the various processes individually, Q sort and factor analysis, and examining them separately as technique and method, respectively. Equally impactful on Q methodology is what Stenner and Stainton-Rogers (2004) termed the Amish-effect, in reference to a small religious sect that is a small, tightknit community of “believers.” Within that community and, as inferred from Stenner and Stainton-Roger’s use of the term Amish, within the Q community, one must convert to join the preexisting society. Within Q, they contend, this requires a substantial epistemological shift before acceptance. Within this context, epistemology is meant to be the relationship between the observer and the observed, which is somewhat turned upside down in Q, where the belief is that only the observed can capture and reveal their subjectivity as represented as their Q sort (Stephenson, 1953b).

Certainly, acceptance of Q methodology’s position within the qualitative–quantitative continuum of mixed methods allows the engagement of a broader community of interest and practice and is not without support within the Q community (Ramlo & Newman, 2011a, 2011b; Stenner, 2011; Stenner & Stainton-Rogers, 2004). And much of the pushback about Q’s position within this continuum is represented as either a misunderstanding that researchers are offering a qualitative–quantitative dichotomy, rather than an interplay of qualitative and quantitative (Ramlo & Newman, 2011a, 2011b; Stenner, 2011), a philosophical rejection of Q as anything other than Q itself (Stenner, 2009), acceptance that Q is qualitative in both practice and theory, despite its use of factor analysis (Brown, 2008), or a belief that mixed methods does not capture the methodological hybridity that exists within Q (Stenner, 2011; Stenner & Stainton-Rogers, 2004). It is important to interject here that Stephenson (1986) alluded to a mix of qualitative and quantitative within his methodology, prior to the dawn of the term mixed methods. Yet before discussions about characterizations, use and misuse of Q methodology can be addressed, the reader needs to understand more about Q.

About Q Methodology

Q methodology consists of a series of interwoven stages that are both qualitative and quantitative. In this section, we will describe the process of performing a Q methodological study, provide a brief historical account of its beginnings, and describe how Q methodology empirically groups people based on their similar views.

The Process of Q

Q methodology studies commence with the development of concourse which typically consist of a collection of statements about the topic that captures a universe of communications relative to that topic. The concourse is selected empirically and represents the conversational possibilities (Stephenson, 1986). A selection from this concourse is chosen that represents these communications in a smaller set of statements called the Q sample. The Q sample often consists of 40 to 50 numbered items that are sorted into a grid with a range of +5 to −5, Most Like to Most Unlike (Brown, 1980) like that shown in Figure 1.

OPEN IN VIEWER

Figure 1.

Example of a grid with a completed Q sort with Q statement numbers included.

The number of items in the Q sample represents the “sample size” rather than the more familiar, quantitative representation of sample size as the number of people participating in the study. In Q, the P-set is the term used for the group of participants. The researcher may try to select a diverse set of participants to facilitate the revealing of different views about a topic; their number, however, is not important. Qualitative researchers would call this purposive sampling as described by Newman and Benz (1998). In some studies, a researcher is interested in the views of a specific small group of people such as a classroom; in these cases, all stakeholders may be asked to participate. Yet another possibility involves researcher interest in a single individual’s views under multiple conditions of instruction such that this person may provide multiple Q sorts (Brown, 1980; Stephenson, 1953b).

As mentioned, participants sort the Q sample into a researcher provided distribution, from Most Like to Most Unlike, to offer a snapshot of the participant’s view (Brown, 1980; Newman & Ramlo, 2010). The Q sort allows participants to make their point of view objective in that it is available to anyone to see (Brown, 1972). These Q sorts are factor analyzed to reveal the operant subjectivity (factor structure that represents the different views as categories of response). The descriptive tables that include representative sorts for each factor, distinguishing statements for each factor, and consensus among pairs of factors are provided through the use of specialized Q methodology software (Brown, 1980; Newman & Ramlo, 2010; Ramlo & Newman, 2011a) such as PQMethod (Schmolck, 2002) which is freeware. These tables, in conjunction with additional data such as postsort interviews or open-ended surveys, are used to interpret, describe, and name the factors/views (Brown, 1980; Newman & Ramlo, 2010). From the development of concourse to the interpretation of the factors/views within Q methodology, there is a purposeful interactive continuum of qualitative and quantitative (Newman & Ramlo, 2010; Ramlo & Newman, 2011a, 2011b). Historically, however, Stephenson’s creation emerged as psychology was moving toward a more quantitative research focus using objective tests and sophisticated statistical analyses. This contrast helped lead to the marginalization of Q methodology, at least in psychology, during Stephenson’s lifetime (Good, 2010).

Faulty Assumptions in Q

Faulty assumptions in Q methodology are most often the result of what we would call nitpicking specific aspects of Q without looking at the larger philosophical epistemological, ontological aspects of the methodology (not method but methodology) as a whole. Q critics have frequently judged these aspects based on quantitative tenets of research. For instance, it is common place for reviewers and others to focus on a lack of validity and reliability within Q methodology. A Q sort offers a snapshot of a person’s view after the individual has operated with the statements. Yet, as Brown (1980) indicates, there are distinct differences between operant subjectivity (as captured in a Q sort) and the more typical scaling and questionnaire methodologies. A person’s Q sort cannot be right or wrong like a scale response. The Q sort is not based on operational definitions or dependent on constructed effects. The Q sort is self-reference. With no outside criterion for a person’s own point of view, validity is of no concern in Q (Brown, 1980, 1993).

Similarly, because all is subjective, the Q factors “are grounded in concrete behavior, are usually reliable and easily replicated, and, happily, are subject to statistical summary which facilitates more careful description and comparison” (Brown, 1980, p. 6). This factor analysis, used to group Q sorts into similar views, is a fundamental aspect of Q methodology. However, once the Q sorts are correlated, the mathematics of factoring is indistinct from that used in R method applications. For this reason, R methodologists are likely to thrust statements about statistical improprieties onto the Q methodologist without understanding that differences arise from methodological considerations (McKeown & Thomas, 2013).

According to Brown (1993), much of the current understandings about Q methodology arose from the debates during the 1930s, 1940s, and 1950s between Stephenson, Burt, Cattell, and others. Stephenson often said that much of the criticisms of Q methodology stemmed from faulty assumptions related to Q’s procedures, philosophy, epistemology, and/or ontology (Brown, 1998; Stephenson, 1954). One of the earliest of these situations, already mentioned in the previous section, is well documented in the article, “Alternative Views on Correlations Between Persons,” where Cyril Burt and William Stephenson (1939) essentially agreed to disagree about their methodologies to group people. If nothing else, clarity about two different purposes and techniques for grouping people were provided in that article where the similarities between Burt’s Q factor analysis and Stephenson’s Q methodology become clarified to reveal how each is actually quite different methodologically. For instance, related to technique, the sources of data are very different—objective data from objective tests and subjective data from Q sorts, for Burt and Stephenson, respectively.

Disagreements have continued about Q methodology, even within the Q methodology community. More general research journals, not specializing in Q, sometimes offer manuscript reviews and/or editorial commentaries that are dismissive of Q. Other times, authors manage to publish articles on Q that are based on various faulty assumptions. An occurrence of the latter happened recently with three publications in Quality & Quantity (Brown, Danielson, & van Exel, 2015; Kampen & Tamás, 2014; Tamás & Kampen, 2015). Authors of the first article, Kampen and Tamás (2014), “Overly Ambitious: Contributions and Current Status of Q Methodology,” claim a review of the Q literature (which is highly limited due to their search criteria) indicates the demise of Q methodology as well as its inability to provide insight into human subjectivity. The authors’ subtitle is taken from a much earlier Q methodology critique by Wittenborn (1961). Similar to Wittenborn, authors Kampen and Tamás (2014) approach their objections to Q from a psychometric framework full of faulty assumptions and a lack of conceptual understanding of the methodology they are criticizing. Certainly, their reference list lacks many of the key contributions in Q methodology over its 80-year history. Omitting key Q publications is representative of the inaccuracies offered by these authors regarding Q methodology. In response, Brown et al. (2015) offered “Overly Ambitious Critics and the Medici Effect: A Reply to Kampen and Tamás.” The Medici Effect is in reference to a resistance to understanding that is comparable to Medici’s stance toward Galileo. The basic tenet of the “Medici Effect” is the insistence of maintaining misunderstandings even when confronted with alternative explanations as well as refusing to seek clarification of the new idea that has been introduced. This is similar to problems with conceptual understanding in physics; many students maintain their prior misconceptions even after effective instruction (Thornton & Sokoloff, 1998). Like physics students who are resistant to change their preconceptions about Newtonian motion (Duit & Treagust, 1998; Ramlo, 2008; Thornton & Sokoloff, 1998), it is not uncommon for researchers to be resistant in accepting Q methodology, especially its philosophical epistemological and ontological aspects. This is the case with Kampen and Tamás (2014) who stress inappropriate psychometric principles to condemn Q methodology.

In their reply to Kampen and Tamás (2014), Brown et al. (2015) offer a brief 80-year historical summary of similar ill-advised critiques of Q methodology. However, the core of their response relates to the philosophical and methodological understandings of Q—from the nature of subjectivity to the development of concourse to the role of factor analysis in Q. It would seem that many of the criticisms of Q are related to its qualitative characteristics, especially, within the mathematical procedures and choices for which Stephenson advocated. Qualiquantology can be uncomfortable (Stenner & Stainton-Rogers, 2004). Certainly, Kampen and Tamás (2014) appear uncomfortable with Q’s hybridity as they offer concerns regarding what they see as unresolvable problems of sampling from a population of statements (selecting the Q sample from the concourse) despite Stephenson’s advocating for using Fisher’s experimental design procedures to create a structured Q sample (Brown, 1980; Stephenson, 1953b, 1986). Similarly, Kampen and Tamás join others (e.g., Cragan & Shields, 1981; R. M. Johnson, 1970) in their concern over what they perceive as inadequately sized P-sets. With a disregard for sample size in Q represented as the population of statements (Q sample), these researchers stress that Q studies require a large number of people (P-set) to conform to their quantitative expectations of this mixed method. As Brown et al. (2015) explain, these are misunderstandings already addressed within the literature, which Kampen and Tamás have overlooked in their zealous attempt to discredit Q methodology. In their response, Tamás and Kampen (2015) dispute the merits of the rebuttals offered by Brown et al. (2015), but offer blame on potentially poor execution of Q studies leading to what they call the dim view, the scientific community has taken of Q methodology with one example of a compilation of qualitative methods that does not include Q.

The book Science, Psychology, and Communication: Essays Honoring William Stephenson (Brown & Brenner, 1972) also offers, surprisingly, examples of misconceptions about Q methodology. Within the opening chapter of that text, Kerlinger (1972), like Kampen and Tamás (2014), takes issue with qualitative aspects of Q and insists on quantitative purity that would move Q to a more objective status which he sees as more desirable. For instance, to make Q’s purpose more objective, Kerlinger (1972) discusses using analysis of variance to analyze Q sorts while stressing the need for tests of statistical significance. Also contrary to Stephenson, Kerlinger insists on structuring Q samples based on constructed effects. Specifically, Stephenson (1953b) stated that “it is a mistake to regard a sample as a standardized set or test of statements, any more than one can hope to regard a particular set of children as a standard sample” (p. 77). Insistence on standardizing a Q sample, as Kerlinger suggests, indicates a misunderstanding of the purpose of the Q sorting process and a push for objective, quantitative approaches only. Overall, Kerlinger’s discussion about the shortcomings of Q, which takes up over half of the pages of his chapter, reveals his misunderstanding of the methodology developed by the person being honored by this text. Kerlinger does acknowledge that Stephenson would not be pleased with some of what he says in his chapter, but this did not stop him from providing his criticisms of and “improvements” for Q. His suggestions and criticisms demonstrate either a misunderstanding or rejection of Q’s philosophy, epistemology, and ontology which has more in common with qualitative research than quantitative, including at its factor analytic stage (Ramlo, 2015).

Frels, Newman, and Newman (2015) suggest that a mixed methods researcher needs to be

respectful of diversity of (a) philosophical assumptions and stances, or the core beliefs within methodology; (b) epistemology, or beliefs about the nature and scope of knowledge; (c) ontology, or issues concerning the nature of knowing such as subjectivity or objectivity; and (d) axiology, or the extent of the role of values. (p. 341)

They stress the need for dialogue about methodological and philosophical choices in support of an inquiry and its purpose. We believe that avoiding such dialogue is based on the “paradigm wars” as described by Greene (2007). However, methodological considerations are frequently at the heart of misunderstandings and faulty assumptions related to Q methodology, most often related to its inherent mixture of qualitative and quantitative which Stenner and Stainton-Rogers described as uncomfortable.

Where Does Q Fit?

Undoubtedly, Q methodology consists of a set of procedures, philosophy, and theory. Some researchers, outside of Q, may only be aware of the Q-sort (the technique of measurement), while others are solely familiar with the use of factor analysis to group people (the method). But it is important to realize that Q is a methodology in the complete sense of that word with technique, method, and philosophical framework. Despite its use of factor analysis, Q methodology has been more accepted within the qualitative community with numerous publications in qualitative annals such as The SAGE Encyclopedia of Qualitative Research Methods (Brown, 2008), Qualitative Health Research (Brown, 1996), and Qualitative Research in Psychology (Watts & Stenner, 2005). Brown (2008) explains that Q methodology embraces goals more typically associated with qualitative research despite its use of sophisticated statistics including factor analysis.

However, Maxwell (2010) describes the use of quantitative data in qualitative studies as controversial; this controversy may explain the frequent downplay of factor analysis in Q, with little “statistical” types of information (even a mention of factor extraction and rotation for instance) provided within many of the articles on Q in qualitative journals, encyclopedias, and handbooks. Conversely, Q has also often been identified as a quantitative method due to its use of factor analysis (Newman & Ramlo, 2010) and a chapter on Q methodology, possessing numerous misconceptions with an overabundance of focus on the factor analytical stage of Q, appears in Nunnally’s (1978) Psychometric Theory.

Q’s acceptance within the mixed methods community is more recent with a key publication as a chapter in Handbook of Mixed Methods in Social & Behavioral Research, Second Edition (Newman & Ramlo, 2010). Q methodology’s identification as a mixed method has met with mixed reviews within the Q methodology community with some accepting it as mixed (Davis & Michelle, 2011; Newman & Ramlo, 2010; Ramlo & Newman, 2011a; Stenner, 2011; Stenner & Stainton-Rogers, 2004). For instance, Davis and Michelle (2011) believe that Q methodology’s hybrid character has “the potential to bridge the gap between quantitative and qualitative methods by helping researchers in these two predominant research styles to fruitfully interact with each other” (p. 562).

Yet other Q methodologists take a strong philosophical stand about Q’s research uniqueness. Paul Stenner (2009) states that Stephenson was resistant to Q methodology being placed within other theoretical frameworks. Other theoretical frameworks could be taken broadly as research frameworks qualitative or quantitative (Stephenson died in 1989 and may not have been aware of the mixed framework’s beginnings), although others have attempted to fit Q within feminism, social constructivism, discourse analysis, and psychoanalysis, among others (Brown, 1993). Stenner & Stainton-Rogers (2004) described Q methodology as fitting into the qualitative research framework of naturalistic contextualization. Yet Q methodology maintains the relationship among themes within the data as it minimizes, compared with typical qualitative research, the impact of the researcher’s frame of reference (Stainton-Rogers, 1995) through the use of complex statistical analysis including correlation and factor analysis (Brown, 1980; Stephenson, 1953b). To further substantiate that Q fits within mixed methods, we need to clarify what is meant by mixed methods and offer reflections on Q’s philosophical, theoretical, and methodological frameworks.

Q Meets Mixed Methods

The current accepted position for mixed methods research is to treat it as a continuum rather than a qualitative–quantitative dichotomy (Newman & Benz, 1998; Ridenour & Newman, 2008; Tashakkori & Teddlie, 1998). Ramlo and Newman (2011a) offer details related to how Q methodology fits into the qualitative–quantitative continuum. Certainly, Q’s mixture of qualitative and quantitative goes beyond simply, for instance, describing the development and selection of the Q sample as qualitative and the factor analytical aspects as quantitative. Such compartmentalization ignores the true hybrid nature of Q (Stenner, 2011). For instance, Ramlo and Newman (2011a) demonstrate that Q methodology can be used to both develop theory (qualitative) as well as test hypotheses to confirm theory (quantitative). Using the Multidimensional Continuum of Research Projects presented by Tashakkori and Teddlie (2009), they demonstrate that Q methodology is most frequently a mixture of aspects from postpositivist view of research (quantitative) and the constructivist view of research (qualitative). Using the methodological continua of QUAL-MIXED-QUANT research of Tashakkori and Teddlie (2009), Ramlo and Newman (2011a) demonstrate that Q methodology fits within their MIXED column.

Mixed research is inherently complex, especially when not compartmentalized into two distinct aspects of research but, instead, combining qualitative and quantitative within a continuous interaction. When a true interaction occurs such that the mixed methods research is integrated, R. B. Johnson, Onwuegbuzie, and Turner (2007) suggest that the research can be identified as qualitative dominant or quantitative dominant mixed method research. In a qualitative dominant mixed method, the qualitative method is primary and supplemented by quantitative concepts or data. This is the situation with Q methodology where even the factor analytical stage possesses a qualitative focus such that factor analytic choices are based on the desire for theoretical significance (qualitative) rather than statistical significance (quantitative; Ramlo, 2015). We would, therefore, describe Q as qualitative dominant mixed method research.

This is similar to what Frels and Onwuegbuzie (2013) suggest that any one quantitative measure might be used to inform a qualitative inquiry and vice versa. Thus, many decisions are involved as qualitative and quantitative research methods are integrated throughout the process of conceptualization, design, implementation, and interpretation. These aspects must be built on the usefulness of the associated methods to accomplish the overarching goals of the study. This complexity makes it more difficult to write about (or propose) a study that is mixed because the researcher must both understand and be able to explain why the choices and processes were selected and carried out along with any other information that supports the use of the mixed methods (Newman, Newman, & Newman, 2010). Without such understanding and explanation, faulty assumptions about the research arise perhaps leading to negative reviewer feedback or published articles that chastise the type of research. The latter is represented in publications about Q that include Cragan and Shields (1981), R. M. Johnson (1970), Kerlinger (1972), and Kampen and Tamás (2014).

The mixed research situation is additionally complex because, as Greene (2007) recognizes for mixed research practice, debates and dichotomies appear to be inherently philosophical and methodological. These debates and dichotomies are “not likely to be resolved any time soon, as they are rooted in different, even incommensurable assumptions and stances about reality, knowledge, and especially the purpose and role of social science in society” (p. 20). Similarly, Denzin (2010) describes the paradigm wars as complex. Methodological, ontological, axiological, epistemological, and epistemic debates engulfed two warring camps of postpositivists (quantitative) and constructivists (qualitative). Within those camps, other disputes erupted that included debates about paradigm “purism” and “pragmatism” within the constructivist camp, for instance.

Onwuegbuzie (2012) suggests that the best position for advocating for mixed methods research is to simply state that “good research is good research, whether it stems from the quantitative, qualitative, or mixed research traditions, as long as meaning ensues that represents interpretive consistency” (p. 195). This position is represented within the Q literature including Stenner (2011). However, in both mixed methods research and Q methodology, debates still ensue which are philosophical and methodological that result in misconceived research and negative manuscript reviews, among other reactions to research that may fall outside of the accepted paradigms of what research is and how researchers should perform their craft. In contrast, less than “worthy” publications can lead emerging researchers astray as far as comprehending research methodologies; these publications may use appropriate procedures, but provide faulty assumptions including theoretical and philosophical aspects of the methodology.

Paradigms

These situations seem connected to the discussion by Onwuegbuzie and Leech (2005) about the qualitative–quantitative paradigm wars. Onwuegbuzie and Leech describe three research camps: purists, situationalists, and pragmatists. They explain that, conceptually, these camps can be considered to be a continuum related to the extent to which each believes that quantitative and qualitative approaches can coexist and be combined. The purists are the most resistant to coexistence/combination of qualitative and quantitative research based on their ideas of research. Kerlinger, Wittenborn, Kampen and Tamás appear to fit into this category with their focus on quantitatively based criticisms of Q methodology. Alternatively, the pragmatists are the most open to coexistence/mixing of methods based on their ideas of research methodologies as tools designed to aid in trying to understand the world (Onwuegbuzie & Leech, 2005). Brown, Stephenson, Stenner, and Stainton-Rogers are examples of pragmatic researchers.

For instance, Stephenson (1986) mocked the reign of objectivity in behavioral and social sciences as obligatory. He insisted that such a stand represents a general misunderstanding of the nature of science. Stephenson (1986) saw objective positivism as an outmoded paradigm while offering Q methodology as something new that “mixes laws, theory, instrumentation and application in what may seem to be hopeless confusion” (p. 39).

We suggest that most faulty assumptions in Q arise from those within the purist camp and that sage Q methodologist defenders of Q ideals are pragmatists as described by Onwuegbuzie and Leech (2005). These Q pragmatists remain steadfast to Stephenson’s (1986) ideas despite what emerging researchers may see as hopeless confusion and/or uncomfortable hybridity (Stenner, 2011; Stenner & Stainton-Rogers, 2004). To prevent emerging Q researchers from accepting faulty assumptions within the literature or creating their own, Q experts and mentors often attempt to assist these emerging researchers to move beyond procedural knowledge of Q such that they also understand Q methodologically, including its “mixed” philosophy, epistemology, and ontology.

Likewise, whether Q or mixed methods, researchers must describe how the selected method(s) best support the purpose of the study (Onwuegbuzie & Leech, 2005). Additionally, in mixed methods research, explanations about procedures (qualitative and quantitative) and their integration must be discussed and understood as well as the revealing of insights made possible through the integration that would not have been revealed otherwise (Newman et al., 2010). Onwuegbuzie and Combs (2010) recognized that “mixing” might occur on various layers of the research process—in addition to integration of data. For this reason, mentoring in mixed research becomes even more critical due to the many decisions involved to integrate methodological issues at multiple intersections of inquiry such as conceptualization, design, implementation, and interpretation (Frels et al., 2015; Onwuegbuzie & Combs, 2010).

Emerging Researchers

Observations by the author have indicated that many Q researchers, while they may efficiently execute accepted procedures within Q methodology studies, struggle with the underlying qualitative–quantitative theoretical framework and philosophy associated with this study of subjectivity. This is further substantiated by an article by Brown (1978) that was written specifically to address these issues relative to Q methodology’s theoretical focus rather than statistical significance. Subsequently, Brown (1998) wrote “The History and Principles of Q Methodology in Psychology and the Social Sciences” for the purpose of explaining the philosophical underpinnings of Q methodology and posted it online for easy access. A link to this manuscript is available on the Q methodology website (www.qmethod.org) managed by the International Society for the Scientific Study of Subjectivity (ISSSS), which is the international Q methodology society.

Support and Mentoring in Q

Certainly, within Q, having texts which provide the philosophical, epistemological, and ontological bases of Q methodology such as Brown (1980), McKeown and Thomas (2013), and Stephenson (1953b) alone have not proven to be sufficient to ensure overarching understanding of Q as a methodology. As Frels et al. (2015) have detailed, researcher evolution requires opportunities for growth that include interactions within a research community. Thus, providing opportunities for dialogue, including mentoring, is important for mixed methods as well as Q methodology. Discourse, such as that within mentoring relationships, helps clarify a variety of issues and assumptions, not unlike the debates between Stephenson and his contemporaries (Brown, 1993). Alternatives, such as offering resources and texts, including conferences and journals, are also helpful but not always sufficient to prevent faulty assumptions including those that appear in print.

The “dialogue” in Quality & Quantity about Q is simply the most recent, published example of faulty assumptions leading to a general misunderstanding of the techniques, methodology, and purpose of Q methodology with a reply that provides some attempt at dialogue about these misunderstandings. The response of Brown et al. (2015) also exemplifies the need for an informed peer-review process where those familiar with a methodology, especially if it is inherently mixed, are invited to provide feedback. In “A Primer for Q Methodology,” Brown (1993) describes the development of his article based on interchanges on the Q-Method listserv where instances of “theoretical and conceptual disagreements about the nature of Q methodology were apparent” (p. 92). Unpublished discourses also take place on the Q-Method listserv, mentioned earlier. The questions and responses on the listserv frequently exemplify the strata within the Q community where those not “gurus of Q methodology” like Steve Brown can be placed in the stages of development for mixed methods students described by Onwuegbuzie et al. (2013): conceptual/theoretical phase, technical phase, applied phase, and emergent scholar phase.

Frels et al. (2015) suggest that access to researchers with appropriate expertise is imperative to design, conduct, and interpret mixed research. Within the Q community, access to expertise is most likely to occur via the Q-Method listserv unless a researcher is fortunate enough to have a Q methodologist at their university or one nearby. Certainly, there are annual ISSSS conferences that take place around the world that also provide opportunities for sharing expertise and answering questions. In the United Kingdom, Q researchers have formed informal “Tea & Q” sessions where mentors and mentees can discuss Q methodology. In Korea, there is the Korean Society for the Scientific Study of Subjectivity (KSSSS), which has a similar purpose as ISSSS but with a greater emphasis on expanding the use of Q in South Korea. KSSSS most recently provided two journals: Journal of Human Subjectivity (in English; recently suspended but with a remaining online presence for previous issues) and Q-Methodology and Theory (in Korean). Certainly, readers can be assured of quality Q methodology studies and theoretical articles within the peer-reviewed journals of ISSSS and KSSSS. Because these journals are not currently indexed, a typical journal database search would not provide these articles; however, both journals are available online. Dziopa and Ahern’s (2011) recent study, “A Systematic Literature Review of the Applications of Q-Technique and Its Methodology,” is symbolic of the problems some find locating high-quality Q methodology articles (both applications and theoretical); the authors performed a simple index search that included the phrase “Q methodology” within the 2008 time frame. Not surprisingly, Dziopa and Ahern did not locate many studies (14) within their database search. However, it is disappointing that they did not seek to enter into a respectful dialogue with Q methodologists to verify this small number of publications represented the complete list of Q publications in that year. Such an undertaking would seem to fit within their stated goals of their study to ascertain the current status of Q methodology research as well as its usefulness. Otherwise, however, Dziopa and Ahern do provide a serious presentation on Q that testifies to its continuing lure for researchers.

Conclusions

Stephenson first introduced the idea of Q in 1935. Dialogue between Stephenson and his dissenters helped shape Q throughout much of the 20th century until Stephenson’s death in 1989. This discourse helped define the procedural, philosophical, ontological, and epistemological of Q methodology (Brown, 1993). Yet, despite how well Stephenson and others like Brown developed Q and published those articles, faulty assumptions (such as, but not limited to, those detailed within this article) have continued to be an issue for Q. Stephenson (1986) offered Q as something new that mixes theory in ways that some may find confusing or uncomfortable (Stenner, 2011; Stenner & Stainton-Rogers, 2004) but which, as he saw it, represented an understanding of the nature of science (Brown, 1980; Stephenson, 1986). Stephenson was correct that some may find this “mix” confusing; confusion has continued into the 21st century, steeped in faulty assumptions and philosophical misunderstandings related to the messy mixture of qualitative and quantitative within Q.

Faulty assumptions and other criticisms of Q have been ongoing since its inception 80 years ago. The emergence of an international society, Q journals, and a Q listserv have enabled a community to grow and maintain Stephenson’s ideas as well as combat these faulty assumptions and criticisms in more organized, constructive ways. The Q community also provides mentoring and substantial resources for all Q researchers, novice, emerging, and established.

Greene (2007) suggested that the mixed methods community could learn from conversations across disciplines and fields of practice. Certainly, the mixed methods community has already set up some of what works in the Q methodology community. There are two international journals (Journal of Mixed Methods Research and International Journal of Multiple Research Approaches) as well as a new society, Mixed Methods International Research Association (MMIRA). Not unlike Q’s ISSSS, MMIRA was created to “promote the development of an international and interdisciplinary mixed methods research community.” Although MMIRA does not have a listserv, they did create an online discussion board (http://mmira.wildapricot.org/forum/1285908) with its first posting—“How can we spread the message or MMIRA across the world so that researchers all over can be interconnected?”

Our response to this question would be that mixed methods researchers and their communities must remain cognizant that there are always novice and emerging researchers and that their struggles are intensified when these researchers must understand a more complicated, messy mixture of qualitative and quantitative—whether it is Q or another mixed method. A new generation of researchers will become true academic scholars if we not only model good research practice but also establish communities that include synergistic mentor–mentee relationships. Emerging researchers, with a push as academics to become published, may find it easier to perform and publish “cookbook” styles of research, where the focus is on following procedures in an algorithmic way. However, seasoned veterans, established in mixed methods and/or Q, must help form a nurturing community that includes open-mindedness and collaboration. Onwuegbuzie and Leech (2005) describe three research camps: purists, situationalists, and pragmatists. Discussions about research paradigms need to take place, while we are also respectful of diversity in research philosophy, epistemology, ontology, and axiology as suggested by Frels et al. (2015). Within Q methodology, misunderstandings and misconceptions about its underlying philosophy, epistemology, and ontology have led to faulty assumptions and published (and unpublished) attacks and criticisms. Conversations about paradigms and hybridization may be uncomfortable, but can better inform research practices including the combining of qualitative and quantitative methodologies. More pragmatic Q and mixed researchers may best explore understanding of our world.