Abstract
Interest in qualitative methods in psychiatric research is steadily growing. There have been several overviews of the principles and methods of qualitative research in relation to psychiatry 1–3 and medicine in general, 4–8 but no practical guides for psychiatrists on how to critically appraise qualitative research. This paper explains qualitative research to clinicians who are more familiar with quantitative methods.
Qualitative research relies on data collected from interviews (semi-structured or unstructured), focus groups, observations, or documents and other written materials. Data analysis is largely inductive, allowing meaning to emerge from the data, rather than the more hypothetical-deductive approach of quantitative research. A qualitative researcher accepts that there are ‘multiple realities’, not just one objective reality. In particular, the ‘truth’ is in the informant's perspective, not that of the assessors. This is very different from quantitative research where the goal is to use objective measures such as tests, structured interviews or questionnaires to reduce bias. A qualitative researcher would say that such objectivity is impossible. Assessment ‘emerges’ from data, rather than being determined ahead of time and the aim is to develop an in-depth understanding of particular cases or circumstances.
However, there are several similarities between quantitative and qualitative approaches. For instance, quantitative concepts such as reliability, validity, statistical power, bias, and generalisability have qualitative equivalents (Table 1). These include triangulation, trustworthiness, saturation, reflexivity and applicability. Reflexivity also shares features of transference. These are described more fully later.
Comparing quantitative and qualitative research
First three rows based on Kuper et al. 6
The first step in reading a paper is to decide what type of study is described. Just as in quantitative research, the particular study design will determine the other features for which readers should be looking (Table 2). Qualitative approaches relevant to psychiatry include grounded theory, participatory action research, case studies, framework analysis and mixed methods. However, as in quantitative research, the design may not always be obvious, or the description given by the authors may not be correct. Table 2 shows the characteristic features of several common designs.
A comparison of selected qualitative research methods
Next, look at the methods section. How was the sampling conducted? In qualitative research, sampling is usually purposive. Study participants are selected according to their capacity to provide data relevant to the phenomenon of interest, and to inform the emerging theory. Examples might include typical or atypical cases, or representatives of particular demographic groups. Rather than excluding outliers or defining tight selection criteria, participants might be chosen for their extreme or discrepant views. Another technique is ‘snowballing’, where participants are asked to nominate someone else who might be interviewed.
Next, assess how the authors collected their data. Were key informant interviews or focus groups used? Aside from greater efficiency, focus groups allow participants to reflect on each other's opinions, thus enhancing engagement, new ideas and data quality. 9,10 In contrast, personal interviews may allow deeper exploration of subjective or sensitive topics and minimize the impact of stronger voices. Did the authors use other data sources such as documents or observations, including the physical layout of the setting, the range of people involved, the activities that occur, physical objects that are present, and the emotions that are felt or expressed? Did the authors consider the Hawthorne effect, the influence the presence of the researchers may have had on the phenomena of interest? Focus groups and interviews should be audio taped and fully transcribed for analysis to ensure accuracy. Emerging themes should be explored in subsequent interviews, re-analysed and further developed in an iterative process until saturation is achieved. This is the point where no new material emerges from subsequent data collection, and resembles quantitative study power.
In qualitative research, reliability and validity are assessed by establishing the trustworthiness of the data. 9,10 Techniques include independent reviews of transcripts and themes, with meetings to discuss any discrepancies and achieve consensus, as well as member checking (a process through which the results are presented to all, or at least some, of the interview participants). Triangulation is another trustworthiness strategy; researchers can only gain full insights into the complexity of the phenomenon of interest by obtaining information from multiple sources and in multiple ways. 3 This can involve triangulation of data collection methods, theories, observers, raters or analysts, and sources such as different times or settings. 11 Further methods of measuring trustworthiness include peer debriefing (to improve credibility), thick description (to explain context), and reflexivity (interpreting findings in the light of how the researchers, and the research process, may have shaped the collected data, including the role of their prior assumptions and experience). Reflexivity encompasses far more than just considering potential conflicts of interest or other ethical issues. It includes the imbalance of power between the researcher and participants, and how the researcher's own beliefs may have influenced the conduct of the study from inception to publication. It can even encompass counter-transference of researcher to participant.
As in quantitative research, there are several commonly used statistical packages for managing and analysing data (Table 1). The most common analytical technique is thematic analysis, which can be used with any qualitative data, especially key informant interviews and focus groups. The aim is to identify, analyse and report patterns or themes within data. 1 Grounded theory analysis is similar but focuses on generating a coherent theory from the data. Grounded theory takes thematic analysis to another level, focusing on generating a coherent theory from the data and integrating themes into meaningful processes. All thematic analysis is guided by the constant comparative method (CCM). 1,12 CCM involves the following steps: (i) collection and analysis of data; (ii) development of tentative conclusions, hypotheses and themes; (iii) collection and analysis of additional data (participant feedback or member checking); (iv) testing of initial hypotheses, conclusions, and themes; (v) seeking of new data sources and perspectives; and (vi) writing the final theory with reassessment as needed. In CCM, induction is used to generate labels for ideas or themes (codes). As codes emerge, deductive processes guide theories about links between concepts and processes.
Other approaches to data include ethnography and phenomenology. Ethnography describes group relations, including the social rules of the setting, and the beliefs, perceptions or knowledge of group members to uncover social, cultural, or normative patterns. Phenomenology focuses on individual participants and explores their beliefs, perceptions or knowledge.
As in quantitative research, the discussion should highlight strengths and weakness in any of the above areas. In addition, transferability (or applicability) to other settings should be considered and is similar to generalisability in quantitative research. Table 2 highlights specific areas to consider for some of the main qualitative methodologies. There are also checklists for critiquing qualitative studies. 13–16
A hypothetical study of staff morale in a psychiatric ward illustrates differences between qualitative and quantitative methods. A quantitative study might consider using measures such as staff turnover, sick leave, number of grievance procedures, and standardized instruments of burnout or psychiatric morbidity. A random sample of staff would be selected and the emphasis would be on objectivity by maintaining a distance from participants. A qualitative researcher would take a fundamentally different approach. They would get to know staff members and their context by sitting in the tea room or on ward meetings. Aside from focus groups or interviews, they would observe the layout of the ward, and interactions between staff, patients or visitors. They would analyse written documents such as ward policies, memoranda, minutes of meetings and even notices on the walls. Staff would be purposively selected for interviews, or focus groups, to maximize the variation in the information provided. Unlike random sampling, they would be selected according to their capacity to provide data relevant to the topic of low morale (i.e., those with highest number of sick days, those who are most engaged or disengaged in the workplace, those who have been in the job the longest/shortest).
If the researchers adopted a grounded theory approach to the study, they would emphasize the collection and analysis of data in steps, with each informing subsequent steps. They would develop a theory about the processes by which burnout occurs and the necessary responses. Action researchers would emphasize the importance of involving ward staff in the design, conduct, analysis and interpretation of the study to find a sustainable and workable solution. The aim of the research would be to remedy the causes of poor morale such as pay, or terms and conditions of service. A mixed methods researcher would use both qualitative and quantitative measures to explore the topic. For instance, if they gathered quantitative data from the organization, they might then use these findings to identify sub-samples for further interviews. Alternatively, if they gathered qualitative data first, they might then use questionnaires to quantitatively test their findings.
Psychiatrists are well placed to use qualitative research as a result of their training and practice. 1 However, more importantly, qualitative research can support a shift from symptom reduction to prevention and recovery for people with mental illness. Qualitative approaches can improve our understanding of patient experiences and the process of therapy, facilitate engagement and partnership between researchers and participants, and promote action for change. 1,17
For psychiatrists to fully appreciate qualitative research, they must be able to appraise that research as they would appraise quantitative research. There are similarities to appraisal, which this paper has described. As in quantitative research, papers should show appropriate sampling, data collection, and analysis, as well as consideration of transferability or applicability. Lastly, ethics in qualitative research goes beyond review board requirements to address complex issues such as confidentiality, reflexivity, and power. Qualitative research can complement or replace quantitative approaches depending on the focus and objectives of the research. An understanding of both approaches is useful in critically appraising the psychiatric literature.
Footnotes
DISCLOSURE
The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the paper.