PROTOCOL: Interview and Interrogation Methods and their Effects on Investigative Outcomes

4a. Criteria for inclusion and exclusion of studies:

The primary products of this systematic review will be two meta-analyses (MA) and subsequent forest plots that graph the magnitude and direction of calculated effect sizes. The first MA (i.e., MA 1) will only include experimental studies in which the “ground truth” (i.e., whether the person is innocent or guilty) is known. The second MA (i.e., MA 2) will include quasi-experimental, field studies in which the ground truth is unknown.

4b. Search strategy for identification of relevant studies:

We will search for published and unpublished, experimental and quasi-experimental studies on information-gathering-accusatory interviewing. Although we will not set an a priori limit on the publication dates of searched studies, we anticipate that the majority of studies that would be potentially eligible (pass the first round of review) will have been published from 1980 to the present.

We have several comprehensive resources to initiate the search. Below are examples:

Justice, B. P., Bhatt, S., Brandon, S. E., & Kleinman, S. M. (2009). Army field manual 2-22.3 interrogation methods: A science-based review [Lists approximately 250 references]

We will also search the following databases:

Criminal Justice Periodical Index

We will use the following keywords to initiate the search. We expect more keywords to be generated as the search progresses. In addition, we will combine keywords to produce more targeted searches, such as “interview and suspect,” and “confession and interrogation.”

Interrogation(ory)

Finally, the reviewers have many well-established contacts with researchers studying interviewing and interrogation here in the U.S. and abroad. In Appendix A, we have started a list of possible researchers to contact. We will reach out to known and unknown contacts for unpublished or ‘in press’ studies to possibly include. We have obtained the programs of the 2^nd and 3^rd (June 2008) International Conferences on Investigative Interviewing. Included in these programs are more than 100 presentations that we can follow up on to determine if the studies have been written up.

4c. Description of methods used in primary research.

First, we describe an experimental laboratory study that may be eligible for inclusion in MA 1. Then, we describe an example of a quasi-experimental field study that may be appropriate for MA 2.

The typical experimental research paradigm on the interviewing of suspects involves first, a mock crime, and second, an interview session. Participants are usually randomly assigned to be guilty or innocent of the crime (or to tell the truth or lie), and then randomly assigned to one of two or more interview styles (or specific interview techniques). Experiments are usually recorded and outcomes are reliably coded.

We use a recent study by Vrij and colleagues (2007) to illustrate. The title of the study was, Cues to deception and ability to detect lies as a function of police interview styles, and published in a leading journal, Law and Human Behavior.

In Experiment 1, 120 college students participated; half of them participated in a staged event (playing the game Connect 4 with a confederate) in which money was taken from the wallet of another confederate. This was the “truth tellers” condition. In the other condition, the “liars” did not partake in this staged event, but instead were given scripted information about the event. The “liars” also were instructed to take the money out of the wallet, hide it on themselves, and pretend to have participated in the staged event.

Next, both liars and truth tellers were told they would be interviewed and to convince the interviewer that they did not take the money. There were three interview conditions; 60 truth tellers and 60 liars were randomly assigned across them. In the “information-gathering” condition, participants were instructed to tell everything they could about the Connect 4 game, providing as much detail as possible, and follow-up questions were open-ended (as opposed to leading). In the “accusation” condition, participants were asked 11 questions with an accusatory tone, such as “Are you sure you're telling me the truth?” and “Your reactions make me think you're hiding something from me.” In the “behavior analysis interview” condition, participants were asked for free recall and then asked 15 BAI questions, such as “Do you think that someone else did purposefully take the money?”

The interviews were then coded and scored using Criterion Based Content Analysis (CBCA) and Reality Monitoring (RM). These scores were used as the dependent measures. In brief, they found that accusatory interviews resulted in no discernible differences between truth tellers and liars when either CBCA or RM scores were examined, whereas the other two interviewing styles did (though not across the board).

In Experiment 2, Vrij et al. (2007) showed the videotaped stimuli (the three interview conditions) to 68 British police officers. The officers were told that they would see clips of interviews of students who were lying or telling the truth. Officers made dichotomous judgments of accuracy, which were used to calculate hits and false positives. In brief, they found that accuracy was unaffected by interview style and that a truth bias was found in that truth telling was more accurately assessed than lying. Effect sizes were reported.

Additional studies have followed similar experimental methods, such as Hartwig et al. (2005) and Vrij et al. (2008 and 2009).

An example of a quasi-experimental study would be Study 2 by Bull and Soukara (2010). In this study, the authors coded 80 actual interviews of suspects, which were randomly selected from a sample of 200 interviews. The authors reliably coded the interviews for the presence/absence of 17 tactics, the extent to which suspects moved towards confession (1 = no change to 5 = move from denial to confession), and a dichotomous confession outcome. The 17 tactics were categorized into information-gathering (e.g., open questions, gentle prods) or interrogatory (e.g., maximization, intimidation, leading questions).

As stated by the authors, “this Study 2 did not find a simple relationship between degree of tactic usage and extent of shift to confessing” (p. 149). However, Study 2 did find that the two tactics of positive confrontation and leading questions (both interrogatory tactics) were more frequent in the interviews producing confessions. These results help to highlight the techniques that produce outcomes (although the accuracy of outcomes are unknown) with actual crime suspects.

4d. Criteria for determination of independent findings

If eligible studies have multiple outcomes, we will compute an effect size (odds ratio) for each outcome measure separately. If studies employ multiple conditions that represent a single construct (accusatorial or information-gathering methods), a weighted mean effect size will be calculated across these conditions, or a single condition will be selected for inclusion in the data. At this point (i.e., prior to our comprehensive search), our understanding of the literature is that most, if not all, studies have not repeatedly interviewed participants over time. Thus, we do not anticipate outcomes from multiple time points to be problematic. However, if we do encounter this, we will use the final outcome to compute effect sizes.

4e. Details of study coding categories

After conducting the searches as described above, a study eligibility code sheet will be developed. This sheet will include basic information about the study (such as author names, journal name and volume, and title) and a checklist for each eligibility criterion described above.

After the subset of eligible studies is finalized, they will be logged and assigned a unique study id number. Again basic information about the type of publication (conference paper, government report) and the study itself (date range of research, subject pool, country) will be documented. Of importance, we will also assess the methodological rigor of the studies. For example, we could use the Scientific Methods Scale designed by Lawrence Sherman, and utilized recently by Cynthia Lum and colleagues (see http://gemini.gmu.edu/cebcp/Matrix/SMSScoreKey.html). This scale categorizes studies into five categories of methodological rigor.

At a broad level, we will code independent, method/procedural, and dependent variables for each study. Here are some possibilities of each. Final variables to be coded will partially depend on the number of studies that examine and report on them.

Independent variables

Interview style: information-gathering (e.g., PEACE), accusatorial, BAI, control (e.g., no instruction group)

Method/Procedural variables

Random assignment

Dependent variables

Confession

4f. Statistical procedures and conventions

We will generate two meta-analyses and a study space analysis.

For both meta-analyses, we will code our outcome variables and assess the distribution of effect sizes for each outcome using a random effects model, including a report of the mean weighted effect sizes, any outliers, and the 95% confidence interval. A weighted analysis (least squares regression) of moderator effects will also be pursued if a sufficient number of studies is present (with sufficient variance among a given set of predictors). A forest plot graphing the calculated effect sizes will be produced.

We will also aim to conduct a study space analysis, which is an analysis that provides a frequency and graphic representation (a matrix) of the current literature identifying the relationship between key independent, classification, methodological, procedural, dependent and measured variables. Of importance, these variables are intersected, and when the study space analysis is completed, the areas which have been ignored or are in need of study become apparent. Thus, whereas traditional literature reviews tend to focus on the findings of scientific studies, study space reviews can highlight what is needed to be studied, which can be quite important for policy-laden issues. An example of a generic study space matrix (from Malpass et al., 2008; Table 1) is below.

As specified by Malpass and colleagues (2008), there are five steps in constructing a study space matrix: 1) identify the studies (which is also part of the meta-analysis); 2) construct for each study a matrix which shows the independent, method, procedural, and dependent variables; 3) identify and incorporate into the matrix constant variables; 4) enter a ‘1’ in the matrix cells corresponding to an intersection of study attributes for each study; and 5) merge the individual study matrices into one matrix. Malpass et al. (2008) conducted a study space analysis for eyewitness identification line-up procedures, and for the effects of alcohol on witness memory. The latter matrix is attached below as an example of a completed matrix.

4g. Treatment of qualitative research.

Qualitative studies will not be eligible for the study space and meta-analyses because they do not meet the experimental or quasi-experimental criteria. However, when such studies are located in our extensive search, we will make determinations about their possible relevance in identifying additional outcome measures, in developing future research questions, and in the interpretation of results. These determinations will be made by logging and coding the studies for content, authors' conclusions, etc. A discussion of any qualitative studies identified in our search will be included in the final report.