Abstract
Setting the scene
This is the second Special Issue of the European Journal of Probation that has its origins in the ongoing work of a European academic network, the COST Action on Offender Supervision, which commenced its work in 2012. 1 The first of the Special Issues, published in December 2014, brought together a collection of articles on the theme of consent and cooperation in the context of offender supervision (see Morgenstern and Robinson, 2014). This was a theme that cut across the work of all four of the Working Groups that make up the COST Action, which address: Experiencing Supervision; Practising Supervision; Decision-Making and European Norms, Policy and Practice. In this second Special Issue, our focus is methodological, and the articles it includes were commissioned from two of the Action’s Working Groups, whose members have been engaged in a common purpose for the last two years or so. That common purpose has been the development and piloting of innovative methods for studying aspects of offender supervision practice comparatively.
To explain the specific origins of our methodological endeavours, it is necessary to revisit a review of the European literature on the practice of offender supervision that we conducted in the first year of the COST Action (Robinson and Svensson, 2013). This review, which collated overviews of research in 15 jurisdictions, reached a number of general conclusions. The first was that, notwithstanding significant variations between jurisdictions, there was surprisingly little empirical research into the ‘everyday’ practice of offender supervision in Europe. We further found that, looking across the studies conducted to date, two methods dominated: namely, interviews and surveys. This led us to conclude that extant research on offender supervision in Europe had rather more to tell us about what practitioners say they do, but rather less about what they actually do. A third finding was that the empirical studies conducted to date were overwhelmingly within a single jurisdiction, such that there was almost no comparative research on which to report. Indeed, we found only one study that included a comparative element in its design (Bauwens, 2011).
Meanwhile, a parallel review of the research literature by members of the Decision-Making Working Group (Boone and Herzog-Evans, 2013) revealed a particular gap in research in relation to the construction of and reactions to breaches of community sentences and post-release supervision. Given the relevance and importance of decision-making in this area across pre-trial, sentencing and release phases – not least for the liberty of offenders – members of the Decision-Making working group resolved to focus their attention on this area and to explore the methodological options for studying breach processes and practices comparatively.
Members of both groups, then, set themselves the challenge of developing novel and less commonly used methodological approaches suitable for the collection of data about aspects of offender supervision practice across multiple jurisdictions, and in this collection we present the fruits of their endeavours. Each of the four articles in this collection considers a particular methodological approach. In the first contribution, Carr et al. discuss their use of visual methods – specifically, photo-documentation – to capture information about the contexts for and cultures of probation work. Practitioners in five jurisdictions participated in a pilot study, each producing their own ‘data’ in the form of photographs that the researchers then sought to analyse and compare. In the second contribution, Boxtaens et al. discuss their attempts to develop a shared framework for the direct observation of probation practice. In this project, researchers from five jurisdictions undertook to observe initial one-to-one supervision meetings between practitioners and their clients, with a view to finding out what is common, and what is different, in these encounters between jurisdictions. Our third article, by Rokkan et al., is an exploration of the development and use of practice diaries: that is, records of the daily activities of practitioners, constructed by the practitioners themselves. Fourteen practitioners from five jurisdictions participated in this pilot study. Finally, in our fourth contribution, Maguire et al. discuss the use of vignettes – scenarios that are designed to reflect potential ‘real life’ cases or situations – to explore and understand how and by whom decisions are made around the breach of community sanctions and measures.
Each of the four articles in our collection seeks to convey the pleasures, pains and challenges of developing and applying an ‘innovative’ method in small-scale pilot projects that all had a comparative dimension, and readers will find many issues, obstacles and dilemmas that were shared across these very different projects. In this introduction, we identify some of the experiences that were common across two or more of the studies and that stand out to us as worthy of highlighting. These fall into three themes, relating to approaching collaborative comparative research; the research process; and analysing and comparing ‘novel’ data. We conclude by addressing the question: why work with collaborative comparisons?
Approaching collaborative comparative research
International comparative research is most often understood as comparisons of policies, the situation in different nations and other ‘big’ issues. Thereby, comparisons are often made by means of large quantitative data sets from registers and databases or from different kinds of surveys (cf. Aebi and Linde, 2015). International comparisons of qualitative aspects of practice are far scarcer.
The practice of offender supervision is, on the one hand, global, as it is a practice performed in many places. On the other hand, however, it is a very local issue as each practice is formed by – and performed within – its specific context. Nevertheless, we know that the practices of offender supervision have some common aspects. For example, they are carried out by workers who are based in organizations. We know that the practices are framed by time, place, regulations, procedures and that the actors involved exercise some degree of discretion in their everyday work to get the system and the changing lives, situations and behaviours of offenders to work together. We also know that, in the practice context, ideas about how to work travel between countries. The same programme, tool or model might be used in many countries (e.g. McFarlane and Canton, 2014; Tonry, 2015). If it is possible to transfer ideas, it follows that the practices that implement them must be comparable.
Having stated that practices of offender supervisions are comparable, the question that inevitably follows is how to go about doing comparison? Farrington (2015) has suggested that there are two main approaches, which he calls the ‘safari’ method and the ‘collaborative’ method: In the safari method, a researcher visits another country, reviews relevant literature and statistics, talks to key persons, and then goes home and compares his or her own country with the country visited […] In contrast, the collaborative method involves cooperation between at least one knowledgeable researcher from each country. (Farrington, 2015: 387)
There are of course arguments in favour of both methods (see also Robinson and McNeill, 2015). Nelken (2000), for example, is an advocate for ‘going native’ as the only way to truly understand another culture; on the other hand, such a method does not necessarily counteract the risk that the researcher may misunderstand what he or she sees, or will be thwarted by language barriers. Farrington (2015) argues that the collaborative method minimises these risks, and we would agree.
A major strength of a collaborative approach lies in its potential to expose the cultural and cognitive limitations of individual researchers. There are, as Smelser (2003) has observed, two contradictory aspects of this: The first is that those who have been socialized into a given culture and society are the only ones who can truly understand it. This position has had a number of different formulations, the most recent of which is ‘standpoint theory’. The contrary aspect of the point is that those who have been thus socialized are blind to the understanding of their own, because so much of it is unconscious, taken-for-granted or not-to-be-discussed – ‘the last creature to discover water is the fish’. In keeping with this version, it took an outsider, de Tocqueville, to develop such insights about America, and in general, foreigners can understand things that natives cannot. On the other hand, foreigners, being foreign, cannot get ‘inside’ in the way natives can, and so must always experience a residue of ignorance and perplexity. The most striking fact about these points is that both are simultaneously right and wrong. That fact, moreover, constitutes to my mind the strongest argument for international collaboration, because it is through that avenue that the strengths of both principles can be maximized and the weaknesses minimized through constant exploration and enlightenment. (Smelser, 2003: 655–656)
Thus, a reasonable way of working with qualitative comparisons, which enables the combination of international aspects with local knowledge of each context, is to work together in collaborative comparative work.
Collaborative work, however, has its own challenges. When scholars work together they combine their perspectives, frames of references and different cognitive bases from their different contexts, and these differences are amplified when researchers come from both different disciplines and different countries. In a European context, where researchers with several different languages work together, language is a big issue. In everyday life, there are an enormous number of situations in which people talk about the same thing using different words, or use the same words to describe different things. This is even more evident when working across jurisdictions and language barriers. Our understanding is filtered through the concepts we use, and thereby by the language we use. We talk, write and collaborate in English, a language that all know, but are to different extents comfortable with. Exchanging ideas in English typically requires us to use the concepts used in practice in Anglophone countries. In doing so, we potentially give those concepts the status of norm. Yet language reflects the penology and organisational structure and the labels used to convey meaning in a particular place (Herzog-Evans, 2013). Language is not only an issue if you are English, Swedish, French, Croatian, and so forth, it is also an issue of roles, traditions, cultures, contexts, etcetera. Pösö (2014) suggests that we should strive to use robust translations that recognise the context-bound particularities, since language is not only an issue between countries, but also between cultures and settings. In her study of child welfare practice, she noted the language of the client, of the helping professionals as well as of contextual values and norms. Then, in doing and presenting research, she adds the empirical language of the researcher, the abstract language of the theorist and the categorical language of the information scientist.
When we translate our local terminology into another language, something is lost. Simultaneously, quite often something is found, because when we work across languages, the questioning of meaning becomes the norm. The challenge in using one common language (the language of COST Actions is English) leads to discussion about ‘what do you mean when you say…’, and it is through these types of discussions that scholars get closer to a genuine shared understanding. The challenge, and the opportunity, in collaborative work is to move beyond the language issues to the phenomena that are being studied. In the articles in this issue we find many examples of how conceptual discussions on issues and variations in the studied practices ultimately resulted in more structured and robust tools for gathering data. This issue is perhaps best illustrated in the article by Maguire et al., in which the authors explain how their research began with a careful discussion of the concept of ‘breach’ across jurisdictions and the pursuit of a shared approach to its operationalisation. Ultimately, all the research teams were motivated by the pragmatic need of being able to compare their findings.
The research process
A well-documented challenge of research in criminal justice settings is gaining access to the field of enquiry, and doing research into practice is no exception (Robinson and Svensson, 2013). If our collection of pilot studies tells us anything new in this regard, it is that access channels varied considerably between the jurisdictions in which we were based. Furthermore, the fact that ours were small-scale pilot studies did not reduce the complexity or formality of access negotiations in some countries; in others, cumbersome official channels were able to be bypassed by virtue of the small scale of our requirements, and/or our existing good relationships with practitioners who were willing to get involved. This means that comparative research designs needs to build in enough time to accommodate both more and less complex access negotiations, and this is most likely to impact on larger-scale research straddling multiple jurisdictions.
There are, of course, several very good reasons why access to practitioners requires negotiation – not least the onus on researchers to satisfy gatekeepers that their research will be conducted ethically – but also because practitioners’ time is a key resource in the organisations that employ them, not to be squandered in encounters with researchers who might take them away from their primary functions and interactions with clients and/or other professionals, without any obvious direct benefit to the individuals or organisations concerned. This is perhaps particularly true of interview-based research that, as we have already noted, dominates the field. In all our pilot studies, we made time (and thus resource) demands on practitioners, the amount varying between studies and the methods they employed. The practice diaries researchers, in particular, were anxious about the time demands they placed on participants, not only because they were asking for both diary entries and reflections upon the process of recording practice via the diary method; but also in light of the context of increasing bureaucratisation in probation work (and thus form-filling and ‘desk-time’) in at least some of the countries concerned (see Rokkan et al.).
However, our studies also revealed that the potential demands or pains of participation could be accompanied and even outweighed by the pleasures or benefits of active involvement of the individual in the process of data creation. In both the visual methods and practice diaries studies, practitioner-participants were largely in control of the data creation process, either in the form of taking photographs or writing diary entries – albeit within structures or frameworks designed by the academic researchers. These then are both methods that take seriously the notion of co-production, 2 and both Carr et al. and Rokkan et al. point in their articles to the potentially empowering effect of giving the practitioner control in this way. They also argue that this chance to participate so actively in the research could have had particular resonance for those practitioners who were at the time of the research experiencing organisational turbulence and/or reorganisation (as several were), when opportunities to feel in control of their environments may have been in short supply (see Robinson et al., 2015, for a UK example of this).
Analysing and comparing ‘novel’ data
All our pilot studies managed to generate some useful data from a number of jurisdictions, and in that regard they can be regarded as having made significant steps toward advancing a comparative agenda for empirical research on offender supervision, a field that is still very much in its infancy (Beyens and McNeill, 2013). However, the generation of data was not the biggest challenge in any of the pilot studies. All of the research teams devote considerable attention in their articles to the task of analysis, and the complexities inherent in comparing data generated by different researchers in different jurisdictions. The majority also consider the added complexities involved in analysing unfamiliar data sets.
It is very interesting to note that all three of the research teams whose methods did not include a face-to-face interview (i.e. the direct observation, practice diaries and visual methods teams) confronted issues around the interpretation of the specific types of data their studies generated. For these research teams, accessing the subjective meanings of participants proved less than straightforward, and this was particularly the case where practitioners were involved in the production of their own data (see above). In the visual methods article, for example, Carr et al. explain that whilst ‘the possibility of a picture “speaking a thousand words” seemed to us to provide a window through which we could explore different cultures and practices of offender supervision’, they were also aware of the ‘seductive simplicity’ of this idea, and the reality that ‘reading’ photographs taken by someone else was unlikely to be a straightforward enterprise. For this reason, they asked participant-photographers to supply some written text alongside the images they produced, and (where possible) conducted a brief interview with participants in which their images were discussed. In a similar vein, Rokkan et al. explain how and why they adopted a ‘diary-interview’ method in their pilot practice diaries project. Thus, a shared finding of the direct observation, practice diaries and visual methods teams was that understanding the subjective intentions and meanings of practitioners, and the particular contexts for their work, was problematic based on data generated by the single method alone. Ironically – given our original intention to depart from interviews as an over-used methodology – all three teams found that a fuller understanding was only possible when the innovative methods were supplemented with ‘traditional’ interviews.
These articles also tend to conclude that in order to enhance our understanding of practice – whether in a single jurisdiction or multiple ones – mixed methods (such as practice diaries combined with observations or photographs) are well worth consideration. This reflects Smelser’s (2003) argument for combining methods in international comparative research, on the grounds that it enables different perspectives to be integrated.
Why work with collaborative comparisons?
In this introduction, as well as in the articles in this Special Issue, readers will find much discussion of the challenges and difficulties involved in working with collaborative comparisons. But it would be remiss of us to fail to mention the considerable pleasure that we have also derived from our experiences of working collaboratively. We think we speak for all of the contributors to this Special Issue when we say that the joys of collaboration have vastly outweighed the pains, and that we have all, at one time or another, felt invigorated by the experience of being ‘fish out of water’: both as relative novices in the arena of comparative research, and in working with research methods that have been largely new to us.
Although all of the pilot studies discussed in this Special Issue were small in scale (and unfunded), we contend that they have much to tell us about the strengths and weaknesses of working with collaborative comparisons, and the methodological insights that are presented in the articles will, we think, be extremely valuable for other researchers who wish to go down similar paths in their research. Through the process of testing ‘innovative’ methods – namely, photo-documentation; direct observation; practice diaries and vignettes – we have learned a great deal about how best to acquire comparable data, and how it is possible to get closer to the ‘coalface’ of practice. That said, as noted above, we think there is much to be learned from comparative, collaborative research that combine two or more of the approaches we have tested in our pilot studies.
It is imperative however that, as we and others move forward, we maintain a critical stance in respect of both attempting collaborative comparisons, and the particular methods we have tried: Social scientists, especially economists and other quantitative researchers, are often rightly accused of resembling people carrying hammers and looking for nails to hit. A precondition to figuring out how to do comparative research effectively is to be clear about why something is to be studied and what is hoped to be learned. The nail has to come first. (Tonry, 2015: 506)
As social scientists who have experimented with some innovative methods, we run the risk of becoming over-enthusiastic about our new ‘hammers’, and becoming distracted from the key matter of identifying appropriate ‘nails’ to hit. Thus, the questions: ‘why work with collaborative comparisons?’, and ‘with which tool?’ are questions that need to remain in the foreground of our future research endeavours.