Generating program theories for a trauma consultancy service in early learning settings: Insights on using realist methodology

Abstract

Realist research is increasingly used to evaluate complex interventions. However, it can be challenging to codify and implement, with few examples to guide the process. This article describes how a team of social care leaders, practitioners and researchers developed initial program theories for the Trauma Consultancy Service (TraCS) in early learning settings, as the first phase of a realist evaluation. It explores conceptualisation of realist terminology, design and facilitation of realist interviews, and data coding and analysis using retroductive reasoning. Qualitative interviews with the TraCS team focused on understanding contextual factors, resources provided by TraCS consultants, changes in educators’ reasoning and how components interacted to generate educator and child outcomes. Eight program theories capture how TraCS supports educators to develop a trauma-informed lens and practice. This research contributes to understanding of the benefits of welfare sector-driven consultancy in early childhood.

Keywords

early childhood education and care (ECEC),trauma trauma-informed consultancy realist evaluation realist methodology program theory

Introduction

Trauma, defined as an emotional response to a terrible event, can have lasting adverse effects on an individual’s mental, physical and social wellbeing (American Psychological Association, 2022; Substance Abuse and Mental Health Services Administration, 2014). For young children, trauma can result from neglect, abuse, bullying and other adverse experiences. It can be acute or chronic, directly experienced or witnessed and overwhelm their ability to cope (Cicchetti & Valentino, 2006). The prevalence and incidence of childhood trauma is difficult to quantify, influenced by the multiple types, definitions and ways of measuring trauma, underreporting and inadequate surveillance (Saunders & Adams, 2014). It is estimated that around 8.9% of Australian children experience physical abuse, 8.6% sexual abuse, 8.7% emotional abuse and 2.4% neglect, though these rates could be higher as such experiences are difficult to measure (Moore et al., 2015). Childhood trauma can have debilitating and long-term effects (Australian Bureau of Statistics, 2019), increasing lifetime risk of psychopathology, including attention deficit hyperactivity disorder, depression, anxiety and personality disorders (De Bellis & Zisk, 2014; Dye, 2018).

Programs to support children who have experienced trauma include clinical, family and community-based approaches (Duffee et al., 2021). However, young children are often underrepresented in intervention efforts (Lieberman et al., 2011). Early Childhood Education and Care (ECEC) may be an important setting to buffer the ill-effects of trauma, and there is growing emphasis on supporting early childhood educators to mitigate and manage risk through trauma-informed practice. Yet, few robust evaluations have been conducted in ECEC settings and evidence for intervention is limited (Bartlett & Smith, 2019).

Realist evaluation is a form of theory-driven inquiry that strives to understand complex social and health problems and strengthen the explanatory power of evaluation studies. It extends examination of whether an intervention works to exploring what works, for whom, in what circumstances and in what respects (Pawson & Tilley, 1997). It is therefore well suited to building theoretical understanding and investigating complexity (Marchal et al., 2012), and has been applied in social care (Tennant et al., 2020) and early childhood settings (Fick & Muhajarine, 2019). However, researchers report difficulty applying the methodology in practice (Dalkin et al., 2015; Marchal et al., 2012). This lack of methodological guidance led to the Realist and Meta-narrative Evidence Synthesis: Evolving Standards (RAMESES) project, which encouraged authors to describe the analytical processes used in realist studies (Gilmore et al., 2019).

This article intends to contribute to the methodological literature by exploring how realist approaches were understood and applied. It describes how a team of social care leaders, practitioners and researchers developed initial realist program theories for a trauma-informed consultancy service in ECEC, as the first phase of a realist evaluation. It begins with a brief background of realist evaluation and an overview of the Trauma Consultancy Service (TraCS). It discusses how the research team operationalised a realist approach, specifically: making sense of terminology, facilitating realist interviews with TraCS consultants, and data coding and analysis. It then outlines the methods used to develop initial program theories. Finally, to articulate how realist methodology was conceptualised and used, it provides a snapshot of the program theories and author reflections on applying the approach in practice.

Realist evaluation

Realist research is underpinned by the principles of realism. Emerging from the work of Bhaskar (2008), critical realism was offered as an alternative to positivism and constructivism (Denzin & Lincoln, 2011), however, draws on both in its account of ontology and epistemology (Fletcher, 2017). Realist evaluation, a form of realist research developed by Pawson and Tilley (1997), is underpinned by scientific realism. Similar to critical realism, it accepts that reality exists independent of the researcher, however, acknowledges that understanding of this reality through science is unavoidably linked to the researcher (Marchal et al., 2012). Drawing on a generative understanding of causality, realist evaluation proposes that observable outcomes are influenced by unseen causal processes and forces, and the contexts in which they occur (Dalkin et al., 2015).

Realist evaluation, therefore, posits that intervention outcomes result from contextual influence and underlying mechanisms, rather than the intervention itself. The extent to which mechanisms do or do not operate depends on the context. Observational data alone cannot establish causal relationships between variables. It is necessary to explain why these relationships exist and how they behave in a system (Dalkin et al., 2015), which requires evaluators to shift from examining program effectiveness, typically based on capturing the average answer, to exploring what it is about a program that is working, for who, and in what circumstances. To do this, realist evaluations examine generative causation within the social world by identifying patterns of interactions (Gilmore et al., 2019). Context-Mechanism-Outcome (CMO) configurations are central to this theory-building process. Pawson and Tilley (1997) proposed that contextual features (C) trigger potential mechanisms, including resources offered by a program (e.g. information, skills and support) that can alter the reasoning of program participants (e.g. values, attitudes and beliefs) (M). The interaction between contexts and mechanisms generates varying outcomes (O), that may be intended or unintended. Examining these mechanisms allows program evaluators to understand how an intervention works within specific contexts, and what factors support or impede outcomes (Gilmore et al., 2019; Pawson, 2006). The analysis heuristic, context (C) + mechanism (M) = outcome (O) enables the iterative process of theory building, testing, and refining. Realist evaluation often spans multiple phases from development of initial theories through to synthesising CMO configurations and emerging patterns within existing higher-order theories (Gilmore et al., 2019; Mukumbang et al., 2016; Wong et al., 2016).

Overview of the Trauma Consultancy Service (TraCS)

Consultation is an effective means to facilitate change in ECEC. Through coaching, training and strategy planning, a consultant (e.g. mental health practitioner) can support educators to promote children’s early mental health (Brennan et al., 2008; Perry et al., 2010), and inclusion (Palsha & Wesley, 1998). Consultation has been associated with improvement in educator practice, attitudes and self-efficacy, educator-child interactions, room climate and children’s social and emotional competence (Brennan et al., 2008; Perry et al., 2010).

There are, however, few consultation approaches that focus explicitly on supporting trauma-informed practice in early childhood settings. Many early childhood educators begin their careers with little or no training on how to identify and support children and families experiencing trauma, or how to manage the impact of vicarious trauma (Bartlett & Smith, 2019). TraCS, driven by the Alannah & Madeline Foundation, is a consultancy service available to Victorian ECEC providers, that aims to make expertise from the welfare sector accessible and meaningful to ECEC settings. Combining the knowledge and skill of the educator and an experienced consultant who provides tailored support, space for reflective practice, coaching and advice, TraCS encourages educators to apply a ‘trauma lens’ to their work. As a consultancy service that responds to presenting need, the combination of TraCS service elements (initial consult, in-session and out-of-session consults, phone consultation and tailored training), delivered by consultants within any given context is determined by the needs of the service, educators and children, with this combination revisited throughout the duration of service provision. Consultants, who have a background in social work or psychology, typically work with educators for a minimum of 12 months to shift the support educators receive and in-turn provide to children impacted by trauma.

Consultant, consultee, child and program characteristics can influence the delivery and benefit of consultation (Susman-Stillman et al., 2020; Wesley & Buysse, 2004). A small body of research suggests trauma-informed intervention in ECEC settings can strengthen educator practice and children’s social, emotional and behavioural development (Bartlett & Smith, 2019). However, the causal mechanisms that underline these interventions remain unknown (Fick & Muhajarine, 2019). Clear frameworks for implementing consultation, improved understanding of key components and guidelines for evaluating effectiveness is needed to enable replication of effective models (Perry et al., 2010; Susman-Stillman et al., 2020).

Operationalising a realist approach

Various strategies can be used singly or in combination to develop initial theories. Abstract theories that informed the intervention, tacit knowledge (i.e. mental models, values, beliefs, perceptions, insights and assumptions) about what is working and why from similar interventions, or extracting tacit theories directly from stakeholders through interviews, brainstorming and program documentation are common (Shearn et al., 2017). This study builds on previous research (e.g. Goicolea et al., 2015; Morris et al., 2022), where data-derived initial theories have been elicited from practitioners. Beginning with those who know the program well (e.g. managers and practitioners), rather than users, allows for the capture of quality information about the underlying theory, broad experience of success and failures, and specific ideas on what it is within the program that works (Manzano, 2016). Capturing educators’ experiences is vital to better understand how program mechanisms may influence outcomes and will be undertaken as part of the next phase of evaluation, however, was not the focus of this early work. The section below describes how the research team applied realist methodology in a way that was pragmatic, meaningful and accessible to the TraCS team. Knowledge production and utilisation is an interactive two-way process between researchers and practitioners. Co-constructing understanding of these concepts recognised and optimised practitioners’ expertise, ontology, and epistemology. (Morris et al., 2021).

Making sense of realist terminology

Scholars highlight the complexity of realist concepts and the importance of recognising ontological positionality (Greenhalgh & Manzano, 2021). Marchal and colleagues (2012) mapped how realist concepts, including ‘contexts’ and ‘mechanisms’ were applied in health systems research. They found inconsistent use of terminology and divergent views on the difference between contexts and mechanisms. Pawson and Tilley define contexts as ‘social rules, values, sets of interrelationships that operate within times and spaces that either constrain or support the activation of programme mechanisms’ (Pawson & Tilley, 1997, p. 70). However, a review of 40 realist studies found only 45% included a definition of ‘context’ and there was significant variation in how this was conceptualised. Two narratives emerged: 1) features that acted as enablers or barriers to the intervention, often representing observable features related to space, place, people and things; and 2) features that are relational, underlying and dynamic, and influence the mechanisms through which the intervention works (Greenhalgh & Manzano, 2021). The research team recognised that TraCS is delivered within diverse ECEC settings, where existing beliefs, attitudes, interpersonal relationships and organisational systems may influence how the service is implemented. Context was therefore defined as:

Anything that sits outside of the TraCS offering, and which influences how consultants engage with educators, and how educators engage with consultants, through TraCS.

There has been similar conceptual discussion regarding mechanisms in realist-informed research (Dalkin et al., 2015; Lemire et al., 2020), and researchers often report challenges in disentangling mechanisms from contexts (Shaw et al., 2018). Pawson and Tilley (1997) described mechanisms as the resources offered by interventions and the ways in which those resources influence participants’ reasoning. Dalkin et al. suggested that disaggregating the concepts of mechanism into its two parts can assist with this differentiation, offering an alternative operationalisation within the CMO configuration: ‘Intervention resources are introduced in a context, in a way that enhances a change in reasoning. This alters the behaviour of participants, which leads to outcomes’, represented as M(Resources) + Context → M(Reasoning) = O (Dalkin et al., 2015, p. 4). This underpinned the research team’s definition of a mechanism, which explicitly emphasised the intangible resources TraCS offers:

A mechanism captures the essence of what is experienced between consultants and educators, within the context in which service is being delivered. It helps us to understand how the delivery of an intervention element by TraCS consultants (resources offered, e.g. expressing unconditional positive regard, offering flexibility) is received and acted upon by educators (change in reasoning, e.g. creating a trusted bond, helping an educator to feel safe). The resources offered may influence why an educator chooses to participate, change their behaviour or internalise new knowledge, and this engagement between the consultant and an educator can activate changes in different contexts.

Finally, outcomes in realist research can result from concurrent activation of multiple mechanisms, which have different effects on different people in different situations, therefore producing multiple outcomes that can be intended or unintended (Pawson & Tilley, 1997). We defined outcomes as:

The intended or unintended outcomes that result from TraCS. Examples may include improved educator knowledge, skill, self-efficacy and well-being, or changes or improvements at the centre-level (e.g. centre-level practices, processes or supports).

Planning and conducting realist interviews

Interviewing is an important method for theory development and testing in realist research (O’Rourke et al., 2022). Unlike constructivist interviews that explore participants’ perspectives, beliefs and experiences relating to a topic, ‘it is the programme’s story that is pursued’ in a realist interview (Greenhalgh et al., 2017, p. 1). Understanding of the ‘real world’ is iteratively defined through participants’ stories about the program. However, the application of this interviewing approach is underutilised and underreported (Mukumbang et al., 2020). Although qualitative interviewing is the most common form of data collection in realist evaluation, it has been afforded little attention with regards to the interview itself, with studies tending to rely on traditional semi-structured methods to explore programme effectiveness and barriers (Manzano, 2016).

Manzano (2016) proposed three phases of realist interviews. First, theory gleaning interviews are exploratory, usually conducted with participants who know a program well (e.g. program architects and practitioners). It is recommended that the interviewers have some background understanding of the program and tentative theories about how it is supposed to work. In theory refining interviews, questions are tailored, and individual cases can be used as prompts to refine theories. Tentative theories can be shared, and participants invited to draw on their knowledge and experience to refine, improve, change or discount hypotheses. Frontline practitioners often provide valuable insights about how well a program works for different participants, underlying mechanisms and intended and unintended consequences of implementation; while program recipients can describe their own contexts and outcomes, and what they attribute those outcomes to. Finally, theories are further refined during theory consolidation interviews, where participants are presented with consolidated theories and supporting evidence and stories (Pawson, 2006) and invited to validate or falsify based on their own experience (Manzano, 2016). Re-interviewing participants at different phases supports deeper knowledge and understanding of the program and its processes (Gilmore et al., 2019).

The ‘teacher-learning cycle’ is a fundamental concept in realist interviews. The interviewer presents theories to participants who are invited to comment, confirm, deny or refine the theory. Importantly, the role of the teacher and learner are fluid, sometimes the interviewer is the teacher (e.g. ‘this is an element of a program theory’), and at other times it is the participant who takes on the teacher role (e.g. ‘this does or does not work in this context from my experience’). The interview therefore evolves into a collaborative discussion about the complexities of the program and its implementation (Manzano, 2016). This study’s approach, interview questions and the research team’s active position within the discussion was informed by this guidance.

Data coding and analysis

Realist analysis is not a separate stage of research, but an iterative process of placing ‘nuggets of wisdom’ (Pawson, 2006, p. 127) within a broader explanatory framework. It relies on retroductive reasoning to identify what is underpinning change. Retroduction draws on both inductive and deductive logic, in addition to insights or hunches, whereby evaluators use their own knowledge, experience and common sense to build and test theories (Jagosh, 2020).

Increasingly, realist authors describe their analysis and synthesis processes (Fick & Muhajarine, 2019; Gilmore et al., 2019), and use of qualitative analysis software such as NVivo to manage complex data. For example, drawing on Manzano’s (2016) three-phase interview approach, Dalkin et al., (2021) used NVivo to code interviews and documents to nodes and child nodes. The authors also used memos to bring structure when drawing on multiple data sources for theorising, create a record of the analytical processes, and increase rigour and transparency. In another study, Bergeron and Gadboury (2020) first loosely coded contexts and outcomes that related to identified mechanism concepts, and then used NVivo’s matrix function to identify CMO connections. These detailed and practical exemplars underpinned the coding and analysis process used in this study.

Approach to develop initial program theories for TraCS

The research team designed a qualitative, multi-stage approach to capture how TraCS works from the perspective of the TraCS leadership team and staff, with a focus on the contexts in which TraCS is delivered and the mechanisms that contribute to positive or negative outcomes for recipients. Data were drawn from a review of TraCS documentation, interviews with TraCS leadership and staff, and regular discussion between the research and TraCS teams.

Review of TraCS documentation

TraCS documentation including the program logic, an outcomes matrix, TraCS manual and training materials were reviewed. This offered researchers insight on the underlying theory of change, which informed the interview guide, enabled collaborative discussion during interviews, and provided a starting point for coding. It also offered a lens to review theories gleaned through the interviews. For example, intended program outcomes for educators and children described in the program logic and outcomes matrix further validated outcomes that emerged through the interviews.

Realist-informed qualitative interviews

Ethics approval to conduct this study was granted by the Monash University Human Research Ethics Committee (ID: 32601), with written consent obtained from participants. An interview guide was iteratively co-designed in partnership with TraCS leaders to explore relevant contexts, mechanisms and outcomes. Two rounds of interviews were conducted. The first aligned with the theory-gleaning phase in Manzano’s (2016) approach and were carried out with four members of the TraCS leadership team, including the architect of the program (LC). The qualitative data collected through these interviews were analysed and initial, ‘loose’, CMO configurations developed. The second round of interviews with TraCS practitioners (n = 13) aligned with both theory gleaning and theory refining phases, focussing on consultants’ experiences of delivering TraCS.

At the beginning of each interview, the purpose of the interview and key concepts of contexts, mechanisms and outcomes were explained. Participants were invited to reflect on the contexts in which they work (e.g. ‘thinking about ECEC and the diversity in the sector, how might you define the different types of centres you work with, and can you explain the differences you have observed?’). Exploration of mechanisms focused on what was experienced between consultants and educators (e.g. ‘without using words that describe TraCS service elements, can you tell me what you do to promote positive outcomes for educators?’), while outcomes considered educator, child and centre-levels (e.g. ‘what do you think changes for educators after they have been part of TraCS?'). In addition to seeking consultants’ perspectives and stories relating to contexts, mechanisms and outcomes, the research team described insights gleaned during earlier interviews and invited participants to consider how it aligned with their experience.

Interviews were conducted by two members of the research team (CB, RB) and a Masters-trained member of Alannah & Madeline Foundation’s Research and Evaluation team, all of whom had previous experience in qualitative interviewing. At least two interviewers participated in each interview, which were conducted via Microsoft Teams and lasted between 60 and 90 minutes. Interviews were audio-recorded and transcribed, and the research team took notes and de-briefed after each interview to discuss emerging contexts, mechanisms and outcomes.

Data coding and analysis

De-identified transcripts were uploaded into NVivo for analysis. Transcripts from the Leadership Team (n = 4) were coded first by two members of the research team independently to increase credibility of the analysis. The remaining transcripts were coded by one author (CB). Each transcript was read several times and initial concepts relating to contexts identified. These were coded as nodes in NVivo. Mechanisms (including both resources offered by consultants and educators’ change in reasoning) and outcomes relating to these contexts were added as child nodes linked to each relevant context concept.

Once all interviews had been coded, data were transferred to an online digital whiteboard (MURAL) to allow for greater flexibility. A CMO matrix was created which enabled visual representation of context, mechanism and outcome themes across interviews. The left-hand column of the CMO matrix included the broad contextual factor, the next column included data on how this context was described by consultants, the third column presented data relating to mechanisms (resources offered and educators’ change in reasoning) and the final column captured data relating to outcomes. The research team met six times over two months (1–3 hours each) to discuss coding and emerging CMO groupings. Researchers worked collaboratively, drawing on retroductive reasoning and earlier document analysis to discuss groupings, propose linkages between contexts, mechanisms and outcomes, and collapse and merge categories. Each initial program theory (CMO configuration) was described in at least three interviews. They were presented, discussed, refined and endorsed by the TraCS team.

Snapshot of program theories

Eight initial program theories (CMO configurations) were developed (Table 1), with the ‘loose’ CMOs described by TraCS leaders confirmed and expanded during consultant interviews. The program theories are based around the broad contextual factors that emerged consistently across interviews. The decision to frame each context in the direction that made it more challenging for educators to engage in the service (e.g. lack of time and space, less capacity to respond to trauma) was made with the TraCS team and reflects the importance of these contexts in the delivery of TraCS. For each contextual factor, the common mechanisms that emerged across interviews are highlighted.

Table 1.

Initial program theories for TraCS.

Contextual factor	+ Mechanism (Resources)	→Mechanism (Change in reasoning)	= Outcome
Educator perceives a lack of time and space to engage with programs	• Be curious and open, and validate educators' knowledge, experiences, perspectives and needs • Deliver program components flexibly, in a way that works for the educator/team • Allow space for educators to reflect and integrate knowledge	• Educator experiences an increased sense of safety • Educator is open to engaging in reflective conversations and learning • Educator feels an increased sense of confidence	• Educators apply and integrate a trauma-lens/trauma-informed practices in their work
Educator has less capacity and/or capability to respond to trauma	• Be curious and open, and validate educators' knowledge, experiences, perspectives and needs • Focus on understanding the root causes of behaviour in children	• Educator experiences an increased sense of safety • Educators are better able to understand children’s behaviour, and their response to that behaviour	• Educators use trauma-informed approaches in their practice • Educators have greater understanding of root causes of behaviour in children and the strategies that can be used to support children impacted by trauma
High levels of child and family vulnerability or need in service	• Be curious and open, and validate educators' knowledge, experiences, perspectives and needs • Deliver program components flexibly, in a way that works for the educator/team • Focus on educators' wellbeing • Support educators to reframe thoughts about vulnerable children and the role educators can play • Role-model co-regulation with children • ‘Gently’ coach educator	• Educator experiences an increased sense of safety • Educator feels an increased sense of confidence • Educator more open to seeing the world through a trauma-informed lens	• Educators apply and integrate a trauma-lens/trauma informed practices in their work • Improved educator-child interactions and relationships • Increased understanding of trauma-informed approach
High level of chaos in the room	• Start with empathy • Start where the educator is at • Apply a strengths-based approach • Support the educator to co-regulate • Role-model co-regulation with children	• Educator experiences an increased sense of safety • Educator is open to engaging in reflective conversations and learning	• Increased awareness and understanding of behaviour as an expression of a child’s experience • Increased educator confidence • Positive impact on child behaviour
Educators appear more defensive or resistant to the Consultant or work	• Start with empathy • Be curious and open, and validate educators' knowledge, experiences, perspectives and needs • Apply a strengths-based approach • Deliver program components flexibly, in a way that works for the educator/team • Allow the time to understand what is happening for educators and children in the space • Create and hold space with the educator • Interact with children to explore with educator strategies for managing behaviours • Role-model co-regulation with children	• Educator experiences an increased sense of safety • Educator feels an increased sense of confidence • Educator is open to engaging in reflective conversations • Educator adopts a growth mindset	• Increased awareness and understanding of behaviour as an expression of a child’s experience • Educators apply and integrate a trauma-lens to their work • Positive impact on child as expressed through their behaviour
Educators feel less supported by their organisation	• Focus on educator wellbeing • Hold space with the educator • Create structures and facilitate processes for educators to connect with leadership	• Educator experiences an increased sense of safety • Educator is open to engaging in learning • Educator adopts a growth mindset	• Increased educator resilience and self-regulation
Space is less fit for purpose	• Allow the time to understand what is happening for educators and children in the space • Allow educators to set the pace and focus	• Educator feels validated and open to working with the Consultant	• New knowledge will be sustained and integrated
Lower levels of educator wellbeing	• Start with empathy • Address the immediate need for the educator in that moment • Create and hold space with the educator • Deliver program components flexibly, in a way that works for the educator/team • ‘Gently’ coach educator • Be patient and celebrate the small positive changes	• Educator feels validated and open to working with the Consultant • Educator prioritises their own wellbeing/self-care	• Improved educator wellbeing • Positive impact on child as expressed through their behaviour

It is beyond the scope of this paper to describe each program theory in detail. However, the first theory is explored here. Several TraCS consultants highlighted how they educators they work with feel they have limited time and space to engage with external services and programs (context). Consultants indicated this may be associated with competing pressures and responsibilities, fatigue and burnout, or a belief they did not need the service. This was exacerbated due to the Covid-19 pandemic, with services experiencing additional pressure due to staff illness, limited backfill capacity and challenges engaging via virtual technologies. All consultants who described this as a contextual factor reflected on the importance of listening to educators and being flexible with timing and pace when delivering the service (mechanism, resource). This increased trust and comfort between the educator and consultant, which encouraged bi-directional discussion and learning (mechanism, change in reasoning), and subsequent greater application of trauma-informed practices by the educator (outcome).

Reflections on applying a realist approach

Realist research begins with theorising, followed by testing of theories, refining and testing again. Through this iterative, collaborative process, understanding of the real world is refined (Manzano, 2016). This realist approach was operationalised to build initial program theories for a trauma consultancy service in early learning settings. There is no single set way to conduct realist research, and this represented both a benefit and challenge. It allowed space for the research team to customise and co-create shared understanding of the research goals and application of realist principles. However, it was also time and resource intensive. Representatives from the research and Alannah & Madeline teams met regularly throughout the process, firstly to unpack the approach, make sense of the principles, and customise the language; secondly to develop intervention questions and facilitate interviews; and finally, to analyse data, verify interpretations and workshop and refine initial theories. Elicitation of initial program theories relied heavily on Alannah & Madeline Foundation’s capacity to engage deeply with the research, and access specialist software, such as NVivo.

The interviews supported positive engagement between TraCS consultants and evaluation experts within the Foundation. Distinct from constructivist interviews that explore whether a program is working and may engender fear of judgement in those delivering a service (Chew-Graham et al., 2002), consultants were invited to take part in collaborative, and potentially less threatening, theory-building processes. This created a shared understanding of what works, for whom, in what contexts and why. To support this theory-building process, interviewers guided participants through both micro (inviting deep introspection to explore relational mechanisms and how contexts and mechanisms interact) and macro-level (asking consultants to take a bird’s eye view across the diverse ECEC settings in which they work) thinking, which some found challenging. Despite this, a benefit of adopting this approach within a social care organisation was its promotion of service delivery fidelity through the development of program level theories. The theorising that occurred during interviews and subsequent team discussions raised staff awareness of the different contextual factors that were informing their practice and encouraged reflection on the different ways that TraCS could influence outcomes across these contexts.

This study has several limitations. Although TraCS documentation was reviewed, a realist synthesis of existing evidence was not conducted prior to the interviews due to time and resource constraints, which may have enabled further refinement of the initial program theories (Pawson, 2006). The interviews generated large quantities of data and candidate theories, which the research team needed to refine. A challenge in creating initial theories is providing enough detail within the theory to accurately reflect the program, but not so much that it becomes overwhelming (Fick & Muhajarine, 2019). For example, initial CMO configurations include broad contexts (e.g. time and space to engage with external programs), but do not reflect the several contextual factors that underpin them (e.g. educator experiencing fatigue and burnout, educator believes there is no need for the service in their room). Finally, there is a risk that bias and subjectivity influenced the expression of theories, that misattributions regarding causality were made, and that all program theories were not captured. However, the multiple rounds of discussion with TraCS leadership and staff provide assurance that the eight CMO configurations capture practitioner perspectives on important program theory elements.

Conclusion

The aim of this paper was to describe how realist methodology was operationalised and applied within a social care setting. Initial program theories relate to how TraCS consultants engage with educators across different contexts, and how this engagement interacts with educator reasoning and educator and child outcomes. Consultants recognised various contextual factors influence how TraCS is delivered through their relationship with educators. The findings suggest that customising the delivery of a trauma consultancy service in early childhood, depending on context, can help bridge the gap between the welfare and education sectors, supporting educators to develop a trauma-informed lens and trauma-informed practice. Realist research offered a means to co-construct understanding of consultancy in early childhood in a powerful way. The analytical framework linking contexts, mechanisms and outcomes enabled greater understanding of the key components of effective consultation, an important step in supporting scale-up and replication (Perry et al., 2010). In the next stage of this research, realist-informed interviews with educators and centre leaders will allow exploration and refinement of initial program theories, while mixed methods data collected from educators, caregivers and observers will provide further avenues to test initial program theories, including teacher and child-level program outcomes. This study provides an example of how realist research may deepen our understanding of early childhood interventions and facilitate innovative and insightful findings different to those from other types of evaluation. The research team hopes that other researchers and practitioners, especially those new to realist methodology, will find practical value from these discussions.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iDs

Claire Blewitt

Yihan Sun

References

American Psychological Association (2022). Trauma. https://www.apa.org/topics/trauma/index

Australian Bureau of Statistics (2019). Characteristics and outcomes of childhood abuse. ABS. https://www.abs.gov.au/articles/characteristics-and-outcomes-childhood-abuse

Bartlett

J. D.

Smith

(2019). The role of early care and education in addressing early childhood trauma. American Journal of Community Psychology, 64(3–4), 359–372. https://doi.org/10.1002/ajcp.12380

Bergeron

D. A.

Gaboury

(2020). Challenges related to the analytical process in realist evaluation and latest developments on the use of NVivo from a realist perspective. International Journal of Social Research Methodology, 23(3), 355–365. https://doi.org/10.1080/13645579.2019.1697167

Bhaskar

(2008). A realist theory of science (Rev Ed.). Routledge.

Brennan

E. M.

Bradley

J. R.

Allen

M. D.

Perry

D. F.

(2008). The evidence base for mental health consultation in early childhood settings: Research synthesis addressing staff and program outcomes. Early Education and Development, 19(6), 982–1022. https://doi.org/10.1080/10409280801975834

Chew-Graham

C. A.

May

C. R.

Perry

M. S.

(2002). Qualitative research and the problem of judgement: Lessons from interviewing fellow professionals. Family Practice, 19(3), 285–289. https://doi.org/10.1093/fampra/19.3.285

Cicchetti

Valentino

(2006). An ecological-transactional perspective on child maltreatment: Failure of the average expectable environment and its influence on child development. In Cicchetti

Cohen

D. J.

(Eds), Developmental psychopathology: Risk, disorder, and adaptation (pp. 129–201). John Wiley & Sons, Inc.

Dalkin

Forster

Hodgson

Lhussier

Carr

S. M.

(2021). Using computer assisted qualitative data analysis software (CAQDAS; NVivo) to assist in the complex process of realist theory generation, refinement and testing. International Journal of Social Research Methodology, 24(1), 123–134. https://doi.org/10.1080/13645579.2020.1803528

10.

Dalkin

S. M.

Greenhalgh

Jones

Cunningham

Lhussier

(2015). What’s in a mechanism? Development of a key concept in realist evaluation. Implementation Science, 10(1), 1–7. https://doi.org/10.1186/s13012-015-0237-x

11.

De Bellis

M. D.

Zisk

(2014). The biological effects of childhood trauma. Child and Adolescent Psychiatric Clinics, 23(2), 185–222. https://doi.org/10.1016/j.chc.2014.01.002

12.

Denzin

N. K.

Lincoln

Y. S.

(2011). The Sage handbook of qualitative research. Sage.

13.

Duffee

Szilagyi

Forkey

Kelly

E. T.

(2021). Trauma-informed care in child health systems. Pediatrics, 148(2), Article e2021052579. https://doi.org/10.1542/peds.2021-052579

14.

Dye

(2018). The impact and long-term effects of childhood trauma. Journal of Human Behavior in the Social Environment, 28(3), 381–392. https://doi.org/10.1080/10911359.2018.1435328

15.

Fick

Muhajarine

(2019). First steps: Creating an initial program theory for a realist evaluation of healthy start-Départ Santé intervention in childcare centres. International Journal of Social Research Methodology, 22(6), 545–556. https://doi.org/10.1080/13645579.2019.1595375

16.

Fletcher

A. J.

(2017). Applying critical realism in qualitative research: Methodology meets method. International Journal of Social Research Methodology, 20(2), 181–194. https://doi.org/10.1080/13645579.2016.1144401

17.

Gilmore

McAuliffe

Power

Vallières

(2019). Data analysis and synthesis within a realist evaluation: Toward more transparent methodological approaches. International Journal of Qualitative Methods, 18, 1–11. https://doi.org/10.1177/1609406919859754

18.

Goicolea

Vives-Cases

Hurtig

A. K.

Marchal

Briones-Vozmediano

Otero-García

San Sebastian

(2015). Mechanisms that trigger a good health-care response to intimate partner violence in Spain. Combining realist evaluation and qualitative comparative analysis approaches. PLoS One, 10(8), Article e0135167. https://doi.org/10.1371/journal.pone.0135167

19.

Greenhalgh

Pawson

Wong

Westhorp

Greenhalgh

Manzano

Jagosh

(2017). The Realist interview. http://www.ramesesproject.org/media/RAMESES_II_Realist_interviewing.pdf

20.

Greenhalgh

Manzano

(2021). Understanding ‘context’ in realist evaluation and synthesis. International Journal of Social Research Methodology, 25(5), 583–595. https://doi.org/10.1080/13645579.2021.1918484

21.

Jagosh

(2020). Retroductive theorizing in Pawson and Tilley's applied scientific realism. Journal of Critical Realism, 19(2), 121–130. https://doi.org/10.1080/14767430.2020.1723301

22.

Lemire

Kwako

Nielsen

S. B.

Christie

C. A.

Donaldson

S. I.

Leeuw

F. L.

(2020). What is this thing called a mechanism? Findings from a review of realist evaluations. New Directions for Evaluation, 2020(167), 73–86. https://doi.org/10.1002/ev.20428

23.

Lieberman

Chu

Van Horn

Harris

(2011). Trauma in early childhood: Empirical evidence and clinical implications. Development and Psychopathology, 23(2), 397–410. https://doi:10.1017/S0954579411000137

24.

Manzano

(2016). The craft of interviewing in realist evaluation. Evaluation, 22(3), 342–360. https://doi.org/10.1177/1356389016638615

25.

Marchal

Van Belle

Van Olmen

Hoerée

Kegels

(2012). Is realist evaluation keeping its promise? A review of published empirical studies in the field of health systems research. Evaluation, 18(2), 192–212. https://doi.org/10.1177/1356389012442444

26.

Moore

S. E.

Scott

J. G.

Ferrari

A. J.

Mills

Dunne

M. P.

Erskine

H. E.

Devries

K. M.

Degenhardt

Vos

Whiteford

H. A.

McCarthy

Norman

R. E.

(2015). Burden attributable to child maltreatment in Australia. Child Abuse & Neglect, 48, 208–220. doi.org/10.1016/j.chiabu.2015.05.006

27.

Morris

Blewitt

O'Connor

Skouteris

(2021). Theorising in the social care sector: Conceptualising theory development by frontline workers as they mobilise knowledge to improve outcomes. Journal of Children's Services. Advandced online publication. https://doi.org/10.1108/JCS-12-2020-0077

28.

Morris

Blewitt

Savaglio

Halfpenny

Carolan

Miller

Skouteris

(2022). Using a realist lens to understand the Victorian Family Preservation and Reunification Response in the first year of implementation—towards a better understanding of practice. Children and Youth Services Review, 143, Article 106663. https://doi.org/10.1016/j.childyouth.2022.106663

29.

Mukumbang

F. C.

Marchal

Van Belle

van Wyk

(2020). Using the realist interview approach to maintain theoretical awareness in realist studies. Qualitative Research, 20(4), 485–515. https://doi.org/10.1177/1468794119881985

30.

Mukumbang

F. C.

Van Belle

Marchal

Van Wyk

(2016). Realist evaluation of the antiretroviral treatment adherence club programme in selected primary healthcare facilities in the metropolitan area of Western Cape Province, South Africa: A study protocol. BMJ Open, 6(4), Article e009977. https://doi.org/10.1136/bmjopen-2015-009977

31.

O’Rourke

Abdulghani

Yelland

Newton

Shafiei

(2022). Cross-cultural realist interviews: An integration of the realist interview and cross-cultural qualitative research methods. Evaluation Journal of Australasia, 22(1), 5–17. https://doi.org/10.1177/1035719X211055229

32.

Palsha

S. A.

Wesley

P. W.

(1998). Improving quality in early childhood environments through on-site consultation. Topics in Early Childhood. Special Education, 18(4), 243–253.

33.

Pawson

(2006). Evidence-based policy. A realist perspective. SAGE.

34.

Pawson

Tilley

(1997). Realistic evaluation. SAGE.

35.

Perry

D. F.

Allen

M. D.

Brennan

E. M.

Bradley

J. R.

(2010). The evidence base for mental health consultation in early childhood settings: A research synthesis addressing children's behavioral outcomes. Early Education and Development, 21(6), 795–824. https://doi.org/10.1080/10409280903475444

36.

Saunders

B. E.

Adams

Z. W.

(2014). Epidemiology of traumatic experiences in childhood. Child and Adolescent Psychiatric Clinics, 23(2), 167–184. https://doi.org/10.1016/j.chc.2013.12.003

37.

Shaw

Gray

C. S.

Baker

G. R.

Denis

J. L.

Breton

Gutberg

Wodchis

(2018). Mechanisms, contexts and points of contention: Operationalizing realist-informed research for complex health interventions. BMC Medical Research Methodology, 18(1), 1–12. https://doi.org/10.1186/s12874-018-0641-4

38.

Shearn

Allmark

Piercy

Hirst

(2017). Building realist program theory for large complex and messy interventions. International Journal of Qualitative Methods, 16(1), Article 160940691774179. https://doi.org/10.1177/1609406917741796

39.

Substance Abuse and Mental Health Services Administration (2014). SAMHSA’s concept of trauma and guidance for a trauma-informed approach. HHS Publication No. (SMA) 14-4884. https://www.samhsa.gov/resource/dbhis/samhsas-concept-trauma-guidance-trauma-informed-approach

40.

Susman-Stillman

Lim

Meuwissen

Watson

(2020). Reflective supervision/consultation and early childhood professionals’ well-being: A qualitative analysis of supervisors’ perspectives. Early Education and Development, 31(7), 1151–1168. https://doi.org/10.1080/10409289.2020.1793654

41.

Tennant

Miller

Costantino

De Souza

Coupland

Fotheringham

Eastwood

(2020). A critical realist evaluation of an integrated care project for vulnerable families in Sydney, Australia. BMC Health Services Research, 20(1), 1–15. https://doi.org/10.1186/s12913-020-05818-x

42.

Wesley

P. W.

Buysse

(2004). Consultation as a framework for productive collaboration in early intervention. Journal of Educational and Psychological Consultation, 15(2), 127–150. https://doi.org/10.1207/s1532768xjepc1502_2

43.

Wong

Westhorp

Manzano

Greenhalgh

Jagosh

Greenhalgh

(2016). RAMESES II reporting standards for realist evaluations. BMC Medicine, 14(1), 1–18. https://doi.org/10.1186/s12916-016-0643-1