Abstract
Scholars have long advocated the use of evidence, particularly quantitative evidence, to guide program improvement efforts in the field of human resource development. Yet, there is also widespread recognition that quantitative data has limitations and that new sources of information are useful. The purpose of this article is to consider the use of qualitative evidence to support program improvement efforts within the field of human resource development (HRD). Specifically, the article describes how HRD professionals might design and conduct empathy interviews, a technique widely used in improvement research, to produce a compelling theory of practice that can be used to support program improvement activities. Empathy interviews are thus positioned as a valuable but under-utilized form of qualitative data in the field. As such, the article describes a design process that considers how to select a perspective, structure the conversation, and analyze data. The completion of this process has implications for the development of HRD theory because it shifts the focus from a program-to user-centered understanding of practice.
Keywords
Scholars have long advocated the use of evidence to measure and improve processes in organizations, including those located in human resource departments. In the field of human resource development, quantitative evidence has often been viewed as the most valuable measure. Many variables, such as employee satisfaction, performance, and compensation, can be measured using information derived through surveys, questionnaires, or exported directly from personnel data systems. This is similar to other fields where hierarchies of evidence of privilege the use of quantiative data. Yet, despite the privileged nature of these kinds of data sources, scholars have increasingly acknowledged that quantitative evidence is by no means a perfect measure and may be subject to considerable measurement error (Gerhart, et al., 2000). Indeed, variable performance in human resource initiatives has fueled speculation that some aspects of program improvement could be better studied using qualitative data, such as information derived through interviews and focus groups (Gerhart, et al., 2000). Thus, one of the pressing questions that now confronts human resource development professionals is what information they should use to improve the performance or impact of human resource development initiatives (Stewart, 1996). Strikingly, there is relatively little guidance in the published literature, particularly about the use of qualitative evidence for program improvement in the field of human resource development (HRD). This has practical, methodological, and theoretical implications for professionals seeking to improve the quality, impact, or performance of human resource development initiatives.
In parallel with discussions about the type of evidence that should be used, academic conversations about the potential uses of continuous improvement have grown (e.g., Bryk et al., 2015), with continuous improvement defined here as an ongoing process aimed at improving organizational performance (Bhuiyan & Baghel, 2005) and encompassing the improvement of programs, processes, services or practices. Indeed, conversations around continuous improvement have become a major focus for organizations and the human resource development professionals working within them (Jørgensen et al., 2007). Since many human resource development processes contribute to organizational performance, senior executives and other leaders have expressed interest in improving this facet of their organizations. Thus, in this paper, I take up the perspective that human resource development processes can be best understood by focusing on the experience(s) of users who are directly impacted by these processes and that qualitative evidence is one way to better understand the user’s experiences. Empathy interviews, which are a structured approach to qualitative interviewing through which an individual seeks to understand an individual’s experiences with a program, process, or service, can serve as an important source of information. The information derived from empathy interviews is unique in that it supports the identification of specific practices that merit further revision or improvement. As such, the information gleaned stimulates improvement activities by responding to the experience(s) of users who are directly impacted by the program, process, or service. Human resource development professionals should thus be adept at collecting information that enables them to identity how to improve processes that may not be directly associated with the organization’s bottom line. This likely means that human resource development professionals need to know how to identify, collect, and interpret qualitative data that relates to the experiences of individuals served or supported by the human resources office. This point has been raised elsewhere in this journal (Lester et al., 2020). This article discusses how to design and conduct empathy interviews, which are used to collective user-centered qualitative evidence in continuous improvement, and how evidence from empathy interviews can be used to support program improvement activities within human resource development.
Two Takes on Improvement Research: Design Thinking and Improvement Science
Discussions related to continuous improvement have grown in prominence within the HRD field as scholars have noted important connections between program-level improvement and user experiences. This discussion has given way to methodological debates about what constitutes improvement research, which is often positioned as a new or emerging research methodology (Rutledge et al., 2022). Rutledge et al. (2022) suggest that improvement research essentially hinges on four guiding principles. To paraphrase Rutledge and colleagues, these principals stress the importance of localized problem-solving, engagement of stakeholders in solving a shared problem, the importance of evidence and inquiry, and recognition of the dynamic context surrounding a particular problem or reform. Thus, improvement research has qualities that make it distinct from other forms of research and therefore privileges particular kinds of evidence. Notably, unlike traditional academic research which tends to produce theoretical interpretations designed to cut across contexts, improvement research seeks to produce theoretical explanations for problems that are central to those most impacted. For example, research conducted using improvement methods in a large urban school district led to the adoption of new evaluation and feedback mechanisms for classroom teachers (Austin Independent School District, 2015).
This orientation is reflected in many improvement research methods, including design thinking and improvement science, which are two of the most common. Design thinking is a perspective with origins dating to the late 1960s (Thienen et al., 2018). Current iterations of this perspective have been developed by scholars at Stanford University, particularly John Arnold (1959/2016). Design thinking has expanded to nearly every field, including business, healthcare, and education. In this article, I take the stance that design thinking is a methodology that can inform a user’s experience through program improvement. At its core, design thinking is concerned with people’s experiences, and it strives to encourage experimentation that contributes to meaningful solutions (Rowe, 1998). Razzouk and Shute (2012) define design thinking as an “analytic and creative process that engages a person in opportunities to experiment, create and prototype models, gather feedback, and redesign” (p. 330). When employed within the context of continuous improvement, this perspective assists professionals in envisioning changes that can be introduced and tested.
Design thinking is a popular approach to improvement research because of its emphasis on producing ideas that are responsive to the user’s needs. Despite its popularity, Pressman (2019) notes that “there is no general agreement on a precise definition of design thinking; there are variations across disciplinary cultures, and different meanings depending on its context” (p. 3). Scholars contend that design thinking can be described as a qualitative methodology (Edeleman & Leifer, 2011). Despite this assertion, design thinking is most often characterized as a process that addresses a particular problem that is relevant to a user or group of participants (Pressman, 2019). This process involves tools, including empathy interviews, that are intended to address problems that are relevant to the user or group of participants.
Many scholars argue that design thinking is essentially a five-step process that moves from understanding the user’s experience to testing changes that seek to improve this experience over time. The five steps in the design thinking process include empathizing, defining, ideating, prototyping, and testing. When empathizing, individuals engaged in design thinking seek to understand another individual’s perspective or experience. This process helps them identify the real issue to be addressed. This issue is clarified in the defining stage, wherein the individual conducting the process seeks to identify what must be addressed to improve the user’s experience. In the next phase, images of possibility are presented that could, if implemented, solve the problem and thereby potentially improve the user’s experience. These images of possibility become prototypes or change ideas. The ideas are then tested and data from the testing is used to iterate (improve) the idea to maximize its impact. What is important to understand is that design thinking is a problem-solving technique that helps individuals see problems clearly, identify possible changes, and ultimately implement these changes in meaningful ways (Boller & Fletcher, 2020).
Improvement science, which has also grown in popularity, also shares a commitment to the generation of user-centered change ideas and rigorous testing (Bryk et al., 2015; Langley et al., 2009). Bryk et al. (2015) defined improvement science as a “methodology that disciplines inquiries to improve practice” (p. 197). The methodology is tethered to questions about what must be known and the most practical means necessary to know it. As Bryk et al. (2015) note, “Improvement generally entails a sequence of inquiries, where the results from each test of change offer guidance for the next test” (p. 16). The aim is to take what is learned through repeated rigorous tests to ensure that the final change that is adopted is truly an improvement. The goal of rigorous testing is to reduce “harmful variation” and improve “overall quality” (p. 13). The end aim, of course, is to ensure that whatever is adopted produces consistent results for the individuals most impacted. Thus, much like design thinking, the focus is on improving the user’s experience.
Improvement science and design thinking often function symbiotically in improvement research and the tools (including empathy interviews) can be used equally well in either approach. Thus, within this paper, I suggest that design thinking and improvement science as complementary methods that can support program improvement. In both methodologies, qualitative data is essential to understanding what users want or need for their experience to improve. More broadly, qualitative data is understood as including data that is written, spoken, or visualized (Saldaña, 2011). For example, qualitative data can include “interview transcripts, fieldnotes, and documents, and/or visual materials such as artifacts, photographs, video recordings, and internet sites” (pp. 3–4). The corpus of qualitative data tends to be vast with the aim of analysis being to identify “essential representations” (p. 4). Because improvement research focuses on the stories or experiences of frontline users, qualitative data produces an understanding about what works, for whom, and under what conditions (Bryk, et al., 2015). This understanding, then, guides the identification of changes as well as the ongoing assessment of their impact.
Given their user-centered focus, improvement research requires the development of an empathic understanding of a user’s experience. As Köppen and Meinel (2015) note, empathy in this context is fundamentally about finding out what users need. This requires seeing the experience of users within a particularized setting. To do so, Köppen and Meinel (2015) suggest that this perspective is elevated through “interviewing of and interaction with the user” (p. 20). Finally, it requires “putting oneself in the position of someone else by tracing the experience of that user’s world” (p. 20). Because of the need to understand another person’s experiences, users of design thinking and improvement science often turn to the use of empathy interviews as a technique for determining what works, for whom, and under what conditions (Bryk, et al., 2015). This technique draws extensively from previous methodological guidance related to qualitative research interviews. Yet, the technique itself has received limited attention from research methodologists. Thus, in the next section, I present one articulation of empathy interviewing.
Conceptualizing Empathy Interviews as Qualitative Evidence
In this remainder of this article, I describe how qualitative data derived through empathy interviews can be used to support provide improvement efforts. Before discussing empathy interviews specifically, a general review of the literature pertaining to qualitative interviews is helpful. This research broadly defines models, methods, and various appraisals of qualitative interviews as a source of research information (Kvale, 1996; Roulston, 2021) Roulston (2020) observes that, “Researchers use interviews to ask questions of individuals, dyads, or groups” (p. 2). Thus, the design of an interview might include a single event (i.e., conducted once with one or more participants) or be structured as a series of interactions (i.e., conducted multiple times over a certain period). The structure of interviews can reflect a high degree of formality (i.e., dependent on a highly structured protocol) or assume a more informal and conversational stance (i.e., fluid interactions about a specific topic). Researchers can conduct interviews in-person, online, or using questionnaires. Increasingly, methodologists have articulated the potential uses of technology for completing interviews, including conducting interviews online (Bauman, 2015) and using social media, such as WhatsApp, to ascertain participant’s perspectives (Gibson, 2022). Common to all types and structures of interviews is their primary interest in learning from an individual about their experience, perspective, or beliefs. The use of interviews rests on a specific epistemological perspective. This perspective presumes that experience “authentic” and that the act of interviewing presents opportunities to “offers the opportunity for an authentic gaze into the soul of another” (Atkinson & Silverman, 1997, p. 305). As such, interviews serve as both a research instrument and can be viewed more broadly as a social practice (Talmy, 2010).
Qualitative researchers widely accept the value of interviews as a primary source of data. Roulston (2021) notes that interviews have become “omnipresent” in contemporary society (p. xix). The prevalence of interviewing is such that it transcends both practice and research and is frequently described as a mainstay form of qualitative data (Paulus & Lester, 2022). Roulston (2019) suggests that the methodological literature offers numerous books on research interviews and the techniques associated with them, including how to ask questions, engage with populations, and what strategies to employ to analyze data. This recognition acknowledges, for example, that “researchers using interviews for understanding the social world are equipped with excellent technology for recording interviews, as well as much knowledge to do with how interview interaction might be examined in detail” (Roulston, 2019, p. 5). The successful interview depends on the skilled facilitation of a “question-answer sequence” (Roulston, 2021, p. 2). This sequence includes how the researcher poses questions, how the participant shares information, and what the researcher does once the information is received. As such, this structure captures the basic qualities of an interview regardless of their design.
Despite widespread acceptance, there is some debate about the reliability of interviews as a primary data source (e.g., Potter & Hepburn, 2005). Schaefer and Alvesson (2020) note that interviews have been viewed with a degree suspicion, especially when they are treated as the primary source of data in a research study. Roulston (2021) asserts that there are five primary critiques of interviews present in the literature. First, scholars charge that interview data is potentially biased and has limited validity. This assertion reflects the view that self-reported information could reflect an individual’s beliefs rather than the realities that truly exist at the time. Second, interview data may be skewed by the respondent’s perceptions about what they do versus what they say they do. This critique hinges on the assumption that recollections often vary, particularly as time passes. Third, qualitative scholars have questioned the rigor of analytic methods associated with interview data. This critique centers concerns about the development of interviews as well as the role that theory plays in this development. Indeed, several scholars have critiqued interviews on methodological grounds, with these critiques relating to the analytic processes used with interview data (St Pierre, 2011; St Pierre & Jackson, 2014). Fourth, there is ample concern that interviews may not be culturally appropriate. This critique reflects the need to be more culturally sensitive and aware in research practices, as well as the use of techniques to maximize the recognition of participant voice (Mazzei, 2016). Finally, there have been critiques of interviews which relate to the treatment of participants (Kuntz, 2015), particularly the tendency to impose identities or belief systems through the research process that may not reflect their preferred identity or beliefs. In sum, these critiques of and challenges to interviews highlight some of the taken-for-granted assumptions about how interviews should be conducted and analyzed. While these critiques are important, this article takes the view that careful research design coupled with clarity of purpose can offset these critiques. This may be particularly true when the aim is to understand an experience that centers in a particular program or context.
Designing, Conducting, and Learning from Empathy Interviews
Despite their popularly, methodological literature on empathy interviews is quite thin. Indeed, this paper may be one of the first to offer scholarly guidance about the development, design, and conduct of empathy interviews from a methodological perspective. Empathy interviews are, in essence, user-centered conversations intended to extrapolate specific perceptions or experiences about a program, process, or service from the vantage point of a specific user. In such interviews, empathy is commonly defined as the ability to understand and clearly see the user’s perspective or experiences despite its differences with that of the researcher. This perspective aligns with prior research in psychology focused on empathy (Batson & Ahmad, 2009). This research has found that there may be eight different types of empathy that broadly describe how individuals see another individual’s perspective and thus demonstrate the ability to respond to it, accept it as distinct from their own, or interact with the individual’s perspective to better understand what they have felt. Empathy interviews thus position qualitative evidence, particularly a participant’s perspectives or experiences, as a valuable source of information that can guide continuous improvement planning broadly and program improvement activities specifically. Indeed, the data obtained through these techniques allow human resource development professionals to formulate understandings of improvement challenges they face, as well as to determine whether and how the introduction of specific changes impacts the individual’s experience. Empathy interviews thus are a useful tool to generate improvement-focused theories about the performance and practices of an organization and its human resource development functions.
Nelsestuen and Smith (2020) describe one approach to empathy interviews, though their discussion is primarily situated within a practical framework and not anchored to the theoretical or methodological literature. Nonetheless, they define an empathy interview as “one-on-one conversations that use open-ended questions to elicit stories about specific experiences that help uncover unacknowledged needs” (p. 59). The aim of the conversation is to probe deeply into the experience(s) so that “diverse lived experiences of people are centered in decisions and actions” (p. 59). Unlike a traditional research interview, where the purpose is to explore the participant’s perspectives, an empathy interview strives to deepen this exploration so that the researcher understands in greater detail why the experience was lived as it was. As such, it focuses on “what” and “why” of a given experience. A researcher conducting such an interview will therefore seek to understand what matters to the participant and why it matters more/less than other issues they might encounter.
Human resource development scholars prepare for an empathy interview in stages, which often includes planning the exploration; identifying perspectives of interest; exploring the user’s experience through a learning conversation; and analyzing and processing the data to identify salient leverage points for improvement. Below, each phase is discussed in more detail. Put simply, however, the phases broadly describe how an HRD scholar identifies the most important or valuable perspective and then engages in a conversation with participants who offer that perspective to understand how the program, process, or service (dis)serves the user. To strengthen the illustration, throughout each section a hypothetical mentoring program for classroom teachers is used. Such a program could be easily reconceptualized to fit another context where human resource development occurs.
Planning the Exploration
The aim of an empathy interview is to capture a detailed understanding of the user’s experience with a particular process, program, or service. Thus, empathy interviews seek to identify what matters, what’s missing, or what could be improved from the user’s perspective about the process, program, or service. These conversations are most often conducted one-on-one, using an interview protocol, with questions that are designed to allow the user repeated opportunities to share their experience. However, it is possible to design empathy interviews collaboratively and/or to conduct them in a manner that allows for questions and understandings to be co-constructed with the users. Because the interview foregrounds the user’s perspective, it’s imperative that the conversation be carefully planned so that it generates a robust understanding of the user’s perspective. In line with other types of interviews, empathy interviews follow a question and answer pattern (Roulston, 2021). What differs is the orientation of the researcher. The researcher enters the conversation seeking to identify – rather than seeking to verify – what issues are central to the user’s experience or perspective. The intent is to press the user to explain in significant detail what contributes to their experience or perspective. This explanation thus will inform the improvement efforts that follow.
To illustrate the power of empathy interviews and qualitative evidence, consider the example of a mentoring program for new classroom teachers, which are common human resource development programs in many public school districts. Analyses of the teacher workforce suggests that a combination of factors has contributed to difficulties recruiting and retaining classroom teachers (Sutcher et al., 2019). Given research suggests that classroom teachers are the single most important predictor of student achievement (Kraft, Marinell, & Shen-Wei, 2016), many school districts are taking steps to improve teacher retention and promote professional growth. Research suggests that mentoring is one of the most effective ways to increase teacher retention, particularly that of new or novice classroom teachers (Ingersoll & Strong, 2011). Indeed, the evidence highlights that teachers who participate in mentoring programs tend to be more committed, express higher satisfaction, and tend to remain in the profession longer (Ingersoll & Strong, 2011). Based on research, effective an mentoring program includes different components, such as opportunities for reflection about their practice and the problems they face in their classrooms (Marcos & Tillema, 2006); individualized mentoring or coaching with a senior or veteran colleague who offers guidance (Kraft et al., 2018); peer coaching (Showers & Joyce, 1996), and instructional resources, such as videos of instructional practice designed to support teacher reflection (Hollingsworth & Clarke, 2017). Thus, in a hypothetical mentoring program, a user (i.e., novice teacher) could discuss any aspect of their experience in the program. As such, each component of the program could be the focus of an empathy interview. Alternatively, a researcher might focus on one aspect of the mentoring program which anecdotal data may suggest is not performing as successfully.
Hypothetically, each component of this mentoring program could be discussed in an empathy interview. However, to acquire the most valuable data, it is advisable that an empathy interview focus on a particular aspect of a program. For example, a researcher could ask about the teacher’s experience with the reflection protocol used in the program, the coach/mentor relationship, the professional learning seminars, or the resources and videos provided to them within the program. This allows the individual to recall or reflect on specific attributes of the program and thus serve to pinpoint where a change idea or new approach might be tested and ultimately implemented. Pragmatically, this effectively narrows the scope of the empathy interview. It allows the user to bound their recollections. And, most importantly, it allows the researcher to identify how a particular component of the program might be improved.
Identifying the User’s Perspective
Characteristics that could be considered when selecting a participant.
Structuring the Conversation
Given the purpose of an empathy interview, the next consideration is to determine how best to structure the conversation with the interview participant to identify what they experienced and how their experience potentially differs from other individuals served. The aim of the conversation is to understand how the individual’s experience is unique and thus a basis for expansion (if positive) and improvement (if negative). Consideration might be given to the participant’s experience before, during, or after their participation and thus it could seek to understand how their unique experience contributed to their perception of the program, process, or service. Consideration could include all facets of a system. Bryk et al. (2015) describe this as a systems perspective, which holds that: …the performance of any social system, whether a hospital, a school, or any other organization, is the product of interactions among the people engaged with it, the tools and materials they have at their disposal, and the processes through which these people and resources come together to do work. (p. 58)
Thus, in an empathy interview, the aim of the conversation is to explore how the individual experiences the system and thus what they do to interact with it. The assumption is that by exploring the interaction with different elements of the system, one can discern which elements are most important, beneficial, or problematic.
Throughout the empathy interview, the researcher orients to the user as a knowledgeable informant and thus strives to withhold their own knowledge of the system to avoid guiding the user. Obviously, whenever discussing a program or service, the temptation for the interview participant is to attempt to report what they think the researcher wants to hear. Thus, empathy interview questions should be structured to illicit an understanding of the users’ experience without predisposing what they might say or leading them toward a particular ideas or perspective. Prompts such as, “Tell me about…” or “Describe a time when…”, among others, create opportunities for the user to recall specific experiences. Through these recollections, the researcher is thus positioned to explore what they saw, experienced, felt, gained, or lost because of the way that the experience occurred. Thus, as a methodological consideration, it becomes important for the researcher to consider how their own positionality and subjectivities (Peshkin, 1988), as well as those of their participants, shape the interaction. This requires the researcher to engage in reflexive practice (Day, 2012) throughout the interview process and during their analysis of the user’s responses. Such practice requires that the researcher consider how interpretations have been constructed within the field, as well as attends to the essential expectation that – through research – these constructions can be questioned (Bott, 2010; Hertz, 1997; Van Maanen, 1988).
The structure for an empathy interview is often conversational in nature and presented as a series of open ended questions. The researcher typically enters the conversation with a set of wonderings but typically refrains from imposing them on the participant. For example, the researcher might want to focus broadly on the participant’s experiences and thus maintain an open-ended stance. This might be presented as a question, such as: “Tell me about a time when. . .” This question solicits information from the participant but does not pre-dispose the participant to a particular facet of the program, process, or service. The aim of the question is to solicit as much of the participant’s experience so that the researcher understands the experience in detail. If the researcher wishes to know about a specific aspect of the process, s/he might introduce the participant to a diagram or schematic of the process, program, or service and ask them to describe their interaction(s) with some aspect(s) of it. For example, this might involve asking: “Looking at the process, can you describe how you have interacted with X. . .” Alternatively, one might ask, “What aspects of this process have been the most challenging for you and why?” A researcher might also point to specific aspects of the process and ask the participant, “Tell me how you feel when I point to X. . .” Throughout the conversation, the aim is to solicit as much feedback and perspective as possible without showing the participant what the researcher believes. Indeed, this is one of the hardest parts of the empathy interview process.
A common mistake in planning an empathy interview is to setup the conversation so that it only produces a superficial understanding of the participant’s perspective. This is often demonstrated by a participant responding to a researcher’s question with, “it was good” without further explanation or expansion. Thus, throughout the conversation the researcher should buttress their initial queries with repeated probes. These probes might include questions such as, “Can you provide a specific example?” or “What did that particular experience look or feel like to you?” Regardless, the aim is to dig as deeply into the users’ experience as possible so that previously undisclosed or hidden experiences can be examined during analysis. When conducting the interview, it can be helpful to think of the conversation as one in which the researcher slowly – but deliberately – peels back the user’s experience beginning with what might be viewed as the user’s surface perceptions. These perceptions are often illustrated by short statements such as: “The program was good” or “The program helped me a lot professionally.” While both statements suggest that the program was beneficial, neither offers a complete or compelling picture that helps the researcher understand why it was so.
Building on this, it’s also important for the researcher to explore the “why” behind the experience or perspective. If a participant says, “I really appreciated X,” it could be tempting to leave this as a statement of fact. The ‘X’ could be the facilitation, the content, the organization, or even the donuts! In response to such a statement, the researcher should follow-up and probe, asking, for instance: “Can you share one or two specific things that you really appreciated about X?” or “What about X did you appreciate?” This enables the researcher to position themselves for deeper investigation, particularly if the reason a participant offers is unexpected or novel. As such, it allows the researcher to move closer to understanding the participants’ experience in richer detail.
Illustrative Empathy Interviews
To return to our hypothetical mentoring program example, let’s consider how an empathy interview might be conducted. In the short example below, the researcher (R) is engaging with a classroom teacher who is positioned as the user (U). They are discussing the support provided by a mentor teacher. R: Tell me about your interactions with your mentor teacher. U: They’ve been good, really helpful. R: What has made them good or really helpful? U: Just, you know, the mentor brings a lot of perspective to the work I am doing and helps me think about how I can do it better. She provides me with practical things that are helpful. R: Can you give me an example of the kinds of practical things she offers? U: Sample lesson plans, books from her professional library. She’s also given examples of worksheets or activities that her students like. Things that she and her team members have developed and used. R: Can you give me an example of how you have used some of these practical things, for example the worksheets or activities that she has provided? U: Sure. I used a fractions worksheet that was really cute with my students at Halloween. It required that they have a small back of candy and then we divided the candy in different ways. So, like it asked them to divide it in halves – half gold wrappers, half brown wrappers. Then it asked them to divide into thirds – so chocolates, suckers, and something else. So, the kids loved that because it involved candy, was about Halloween, but I liked it because it was about the helping them see the fractions visually and also having to think about how different amounts represented the fraction.
The passage above illustrates how a researcher moves from high-level questions (i.e., Tell me about…) to questions that are more narrowly focused and designed to generate a deeper understanding of the experience (i.e., What has…; Can you give me an example…). This practice reflects a very familiar question and answer structure, which has been discussed elsewhere in greater detail (see Roulston, 2021). What should be clear from this short excerpt is that the researcher moves from the surface level perceptions toward understanding in very specific and tangible terms what the mentor teacher offered and how it was used. In this hypothetical example, the researcher surfaces that the practical resources provided by the mentor allowed the classroom teacher to make adjustments to their classroom practice and to better support students.
Empathy interviews can be equally, if not more, powerful when a user did not view an experience favorably or encountered some form of difficulty. These experiences serve to illuminate opportunities for improvement or problems that might be addressed through design thinking or improvement science. Consistent with other scholars, a problem is defined as “a gap between the current state and the ideal or aspirational state that requires both further investigation and targeted solutions” (Hinnant-Crawford & Anderson, 2022, p. 297). Continuing the example above, let’s consider what could happen when an experience was not favorable. This hypothetical example illustrates how the researcher (R) might explore an experience that left the user feeling unsettled. The conversation demonstrates how one might move to clarity the issue and identify possible changes that could be implemented. R: How have you worked with your mentor this year? U: Uhm, not as well as last year. R: Can you talk with me a little bit about that? U: I don’t want to be critical, but it just has not gone well. R: What about the experience has not gone well? U: Well, we’ve had some conflict about the kinds of things that I need to feel more supported. She just doesn’t listen to me. R: Ok, what has contributed to the conflict? U: She doesn’t understand what’s happening, the pressure, the need to boost these scores quick. It feels like she operates with a different mindset or doesn’t see reality. R: Can you give me some examples? U: Yes, last week, I had a meeting with my team, and we all felt bad because our students weren’t performing as well as we wanted them to. We all felt that way. So, I shared that with her and she said we should talk about it. We did. But it was like she didn’t understand why these students were struggling. Couldn’t relate. The majority are English Language Learners, and she was clear that she hadn’t worked with ELL students as much in her time because her school wasn’t like ours. It just made me feel like she didn’t see or understand. R: I see. So, what would have helped you feel more supported in that situation? U: Just more empathy or caring about the students, recognition that they were struggling because this isn’t their first language and that’s real and it’s a barrier for the students when the test isn’t written for them. R: What would that look like to you? U: Uhm, her asking me more questions about the students we serve, their needs, the kinds of challenges they have. You know, things that would show she cares more about addressing their needs or helping me do that better than just focusing on the score or their performance, which we know isn’t as important.
In the above example, the user (i.e., teacher) identifies an aspect of the mentor’s practice that did not seem as supportive as she wanted. The mentor lacked both contextual knowledge of the school as well as an appreciation for the unique needs faced by these students. Because of the researcher’s probing, they were able to identify this as an area for improvement and could conceivably (re)design training, support, or resources to help the mentor better connect with the teacher and their unique circumstances.
While both examples are illustrative, they show the power of exploring the user’s experience as a basis for improvement. In both cases, the researcher’s questions moved the user from general concerns and assertions that could not be acted upon toward a specific, actionable understanding that the researcher could address through a design thinking exercise. In doing so, the researcher’s understanding of the user’s experience increased and thus enhanced their position to design changes that addressed their core needs, concerns, or interests. This is the essence of improvement work. Indeed, as Bryk et al. (2015) note, designers and those invested in program or service improvement seek to: …observe people as they carry out their work; (2) understanding how contextual factors shape this work activity; (3) visualize how individuals might engage with new tools and routines; (4) develop, evaluate, and refine changes in prototypes based on users’ experiences with them; and (5) exploit the insights generated through these processes to engineer better goods and services for use effectively at scale. (p 30)
The above quote illustrates the basic principles that guide improvement activities. Situated within the context of an empathic understanding of the user’s reality, improvement leaders can grapple in more profound ways with the cause(s) of poor performance and, in turn, promote solutions that are responsive.
Generating Theories of Practice through Analysis of Empathy Interview Data
To move toward actionable insights, empathy interviews and the data that comes from them must be examined and analyzed. It’s never sufficient to simply record the empathy interview and leave the data to the side. Rather, one should use appropriate qualitative techniques, such as coding (Saldaña, 2021), and analytic strategies, such as thematic analysis (Braun & Clarke, 2006; Lochmiller, 2021), to make sense of the perspectives and claims offered. While analysis could use any number of analytic techniques and employ a variety of strategies, the basic aim is to understand and identify patterns, repeated occurrences, or conditions that explain why the user’s (or a group of users’) experiences is remarkable and how it can inform the researcher’s understanding of the real problems confronting the program, process, or service. Once identified, this understanding then helps the researcher begin to envision changes (or improvements) that could be introduced to adjust the experience. While it is beyond the scope of this article to fully explain how to do qualitative analysis, it is nonetheless important to acknowledge that analysis is an essential step in moving from data to theorized interpretations that can serve as the basis for the improvement effort.
Indeed, it through the analytic process, that researchers produce theories that define problems in the human resource program, process, or service that the user described. This theoretical understanding differs from traditional academic theories because the phenomenon described is situated within a particular context, relevant to specific users, and related to an aim that has significance that may not be widely spread. As Bryk et al. (2015) observe, such theories may be a “small interrelated set of hypotheses about key drivers necessary for achieving an improvement aim” (p. 199). Such theories do not rely on broad theoretical perspectives but are dependent upon observations derived from the exploration of the user’s experience, the researcher’s interpretation of the program, process, or service where this experience occurs as well as the specific improvement aim or goal associated. Thus, by producing an understanding of the experience of the users, HRD professionals generate theories about a practice, process, or program that can be improved by introducing changes, testing them, and ultimately selecting those that work best. Indeed, this is where the novelty of improvement theories emerges and why qualitative evidence emerges as a valuable alternative in HRD improvement activities.
Moving from Qualitative Evidence to Theories of Practice in the Field of HRD
At a practical level, this article has described the design, execution, and analysis of empathy interviews as well as the process of conducting empathy interviews. This technique fits with design thinking (Rowe, 1998) and improvement science (Bryk, et al., 2015). Throughout the article, the discussion has illuminated the value of qualitative data (generated via empathy interviews in particular) in the field of HRD evaluation as well as the potential that improvement research, as a new and emerging methodology, has to the long-term evolution of the field. Throughout the article, a hypothetical mentoring program has illustrated how improvement research can inform the improvement of a generic human resource development intervention. This intervention broadly supports the goal of improving employee retention and performance. Indeed, this goal is widely shared in many professional settings. By showcasing the link between qualitative data and program improvement, the article and method it discusses presents an opportunity to acknowledge the limitations of quantitative evidence in the field (Gerhart, et al., 2000). Notably, it has demonstrated the limitations of relying solely on quantitative measures to assess programmatic performance. Through this demonstration, this article has highlighted how qualitative information can be used – indeed should be used - to improve the performance or impact of human resource initiatives (Stewart, 1996). As such, I seek here to respond to the concern raised by Rousseau et al. (2008), by highlighting techniques that allow practitioners and scholars to better utilize scientific evidence to make effective choices.
Theoretically, this papers makes a contribution to the field of HRD, as well. By highlighting the value of qualitative evidence and the utility of empathy interviews as a data collection technique, this piece advances novel understandings bout the ways in which perspectives of users or program participants can be centered to improve a program, process, or service. This forges new ground in terms of understanding the epistemologies that have dominated HRD research, many of which have separated theory from practice. Indeed, Raelin (2007) observes, The dominant empiricist epistemology governing our educational enterprises in higher education as well as in corporate training and development leads us to separate theory and practice in an aspiration to define the best conceptual models to map external reality. But this brand of, call it “academic” epistemology, often cannot prepare us for engagement any better than classic trial-and-error. (p. 496)
By centering user-defined problems within the realm of program improvement, this concerning divide between theory and practice can be reduced. This recognizes that the “focus and use of theory… in improvement research… is different from its role in other forms of research, where the development of theory is often an endpoint unto itself” (Eddy-Spicer & Penuel, 2022, p. 23). By centering the experiences of participants and users, qualitative evidence moves the field of HRD toward participation in “real design work in generating, selecting, and validating design alternatives at the level at which are they consequential for learning” (diSessa & Cobb, 2004, p. 77). This shift alters how information is and what information counts. Moreover, it compels HRD professionals to consider the agency of participants but the role that their agency plays in shaping the effectiveness or impact of the program, process, or service about which the researcher is most concerned.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
Author Biography
Chad R. Lochmiller is an Associate Professor of Educational Leadership in the Department of Educational Leadership & Policy Studies at Indiana University Bloomington. His research focuses on leadership and policy issues, particularly those relateed to school improvement.