Let’s Talk About Sex: A Systematic Review of Sex-of-Interviewer Effects on the Survey Response

Abstract

This article presents a qualitative systematic review of sex-of-interviewer effects on survey responses on different topics. Overall, we identified 90 scientific studies including 100 datasets between 1962 and 2022. We find that sex-of-interviewer effects are present but also that the topical scope matters. Unsurprisingly, sensitive questions relating to topics such as gender, health, or sexuality, are more susceptible to sex-of-interviewer effects compared to factual items such as demographic questions. We conclude that interviewer sex deserves further attention in scientific studies, as interviewers may affect the survey estimate. We also note that survey reporting on methodological specifications needs to be more transparent to allow evaluating the quality of the survey data collection. Our findings have implications for researchers working with interviewer-administered survey data, as they should consider that interviewer sex can influence survey estimates and, as a result, may wish to control for potential effects.

Keywords

total survey error enumerators gender survey data quality survey methodology

Introduction

Survey data quality is a major concern for all stakeholders aiming to make inferences based on survey data. The Total Survey Error framework (TSE, e.g., Groves et al., 2004; Groves & Lyberg, 2010; Weisberg, 2009; Andersen et al., 1979) suggests that amongst other aspects, interviewer error may affect the overall quality of the survey estimate by inducing measurement error (but see e.g., West & Kreuter, 2015, for the impact of interviewers on nonresponse adjustments and representation error). As such, studying interviewer effects has been one focus in survey methodology and prior research has confirmed the existence of interviewer effects in interviewer-administered data collections (Beullens & Loosveldt, 2016; Davis et al., 2010; Fowler & Mangione, 1990; Groves, 1989, 2004; Loosveldt, 2008; Sturgis, 2014; West & Blom, 2017). Looking specifically at the increasing number of studies on sex-of-interviewer effects, a systematic review of these influences seems useful to understand the overall presence and influence of potential interviewer error. Moreover, systematically reviewing this literature may also reveal what other survey characteristics might influence sex-of-interviewer effects. Previous attempts to analyze the literature on sex-of-interviewer effects are limited, as they followed a narrative review approach (Davis et al., 2010; West & Blom, 2017).

Thus, the goal of this study is to examine sex-of-interviewer effects on the survey response by means of a systematic review, methodically including all eligible studies that fulfill rigorous inclusion and exclusion criteria. In other words, while previous literature reviews included a sample of relevant research in this topic, this review aims to assess the population of studies on the topic of interviewer sex effects on responses. The present work summarizes the modal findings qualitatively and neither offers computations nor standardizations of effect sizes related to interviewer sex error. Further, this article also looks at core study characteristics, including but not limited to sampling strategies, sample sizes, number and sex of interviewers, and sex-of-interviewer—sex-of-respondent matching. Finally, we also account for the topical scope of each study and dataset to identify content areas that may be more prone to sex-of-interviewer effects.

The article is structured as follows: we begin by building a theoretical framework by evaluating the role of interviewer effects within the TSE framework. Next, we discuss the design of our systematic review. Subsequently, we present the results of our analyses organized by looking at the overall study characteristics, first, and reviewing sex-of-interviewer effects organized by topic next. We close with concluding remarks as well as implications of this research for survey methodology, in general, but also for researchers, practitioners, and other stakeholders working with interviewer-administered survey data.

Interviewer Effects and the TSE Framework

The TSE framework (Andersen et al., 1979; Groves et al., 2004; Groves & Lyberg, 2010; Weisberg, 2009) posits that errors can occur at every stage of the survey process and affect the quality of the survey estimate. In particular, it distinguishes representation from measurement error. While the former refers to error induced especially by sampling strategies, the latter focuses on all aspects that affect the questionnaire design and survey response. Interviewer effects predominantly manifest themselves in the measurement error strand,¹ which can be explained looking at the response process model (e.g., Tourangeau & Rasinski, 1988) with four cognitive processing stages: comprehension of the survey question, retrieval of relevant information, use of that information to make the required judgments, and mapping the answer onto response categories for reporting. Prior research suggested that survey interviewers may affect the survey response (Hox et al., 2004) in three ways: 1) their socio-demographic characteristics may (un)intentionally trigger certain memories or responses in respondents; 2) interviewers’ personality traits and social skills may be used more or less successfully to elicit answers; and 3) interviewer opinions and attitudes may shift respondent answers in particular directions. While the standardized survey protocol discourages interviewers from voicing opinions or attitudes, we acknowledge that conscious or subconscious human error may occur, and interviewers do deviate from the survey protocol. This might further depend on the type of questions asked (e.g., Schnell & Kreuter, 2005). Personality and social skills should be important to convince the prospective survey respondents to take part in the survey interview but should play a subordinate role in inducing measurement error. While we acknowledge that interviewers with certain personality traits and social skills may create a more comfortable survey situation in general, the latter two mechanisms are difficult to measure and, thus, beyond the scope of our study. Instead, the focus of this research is on interviewers’ sex² as one of many socio-demographic characteristics that have been found to be influential.³

Social attribution theory posits that respondents may amend their answers in response to notable interviewer characteristics, such as their sex (Fendrich et al., 1999; Heeb & Gmel, 2001). One underlying mechanism might be social desirability bias, i.e., respondents’ tendency to overreport desirable behavior and attitudes and to underreport undesirable ones (e.g., Krumpal, 2013; Phillips & Clancy, 1972; Tourangeau et al., 2000) as a reaction to the interviewer’s sex and thus provide responses that they believe the interviewer may expect, prefer, or tolerate (Stojmenovska & Steinmetz, 2017). For instance, we may expect an impact of interviewer sex on survey responses when respondents use cues, such as social norms, values, beliefs or stereotypes (Fendrich et al., 1999; Heeb & Gmel, 2001). To give an example, if asked about their attitudes on gender equality, respondents might report more progressive views to women in comparison to men. This might occur because of image management (e.g. to appear likeable) or adhering to social etiquette (e.g. to avoid insulting or upsetting the female interviewer).

Prior studies have found that formal question characteristics may offer interviewers some leeway to influence the survey response (Fellegi, 1964; Hyman, 1954; Schnell & Kreuter, 2005): for instance, open-ended, attitudinal, difficult (Mangione et al., 1992) or sensitive questions may be more prone to interviewer effects compared to close-ended, factual, easy or non-sensitive questions (Schnell & Kreuter, 2005). While our systematic review does not break down each study to individual questions, we apply these ideas to wider survey topics. For instance, we may expect that a certain topical scope, such as health, sexuality, or gender attitudes, may be more susceptible to sex-of-interviewer effects compared to factual items, e.g., demographics.

Study Design

Selection Strategy

We follow the guidelines by Moher et al. (2009) on the Preferred Reporting of Items for Systematic Reviews and Meta-Analyses (PRISMA) and present a flowchart in the Figure 1 outlining our study selection strategy.

Figure 1.

Simplified PRISMA Chart for Sex-of-Interviewer Effects on the Survey Response

We began by defining the overall scope of this study. We included all scientific papers that relied on survey data published after 1962, as this time point marks the theoretical and methodological milestone in the field of interviewer effects with Kish (1962). Pragmatic criteria were language and access: research papers had to be written in English and accessible by the authors⁴ (see Table 1, top row).

Table 1.

Study Selection Criteria

Inclusion criteria	Exclusion criteria
Identification
• Study published after 1962	• Study published before 1962
• Language of study is English	• Language of study is not English
Screening
• Real survey data	• Non-survey data (e.g., semi-structured interviews but also job or police interviews)
• Any producer (e.g., government, science, other)	• Simulated data
• Any survey design	• Falsified or fraudulent data
• Survey data	• Unit of observation other than individual survey respondents (e.g., businesses)
• Unit of observation is individual survey respondents	• Only one interviewer sex
• Both interviewer sexes	• Self-administered mode without (virtual) interviewers
• Any interviewer-administered mode (including virtual interviewers)
• Any topic captured in a survey context
• Either or both respondent sexes
Eligibility
• Reported outcome is the response to one or more survey items (including batteries, composite indicators, or indices)	• Reported outcome refers to response styles, response accuracy, item network size, nonresponse and other outcomes
• Response given by respondents	• Proxy answer (interviewer observations or other)
• Study reports at least one numerical empirical outcome	• Narrative findings only, with no numerical results

We used an inclusive search string to search bibliographic databases (e.g., Web of Science, EBSCOhost, and JSTOR). The search also included grey literature websites and portals and manual collection looking at prior review articles, studies, and conference programs. In addition, a call for published or unpublished studies initiated by the authors and distributed through relevant mailing lists allowed finding additional literature. By January 2022, we identified n_s = 1,670 studies; n_s = 456 were duplicate records, which were removed from our database, leaving us with n_s = 1,214 studies included for screening.

At the screening stage, we applied clearly defined eligibility criteria summarized in Table 1 (middle row). We focused on the use of real survey data, as opposed to simulated, fabricated or fraudulent data, that were collected using an interviewer-administered mode⁵ and had an individual as the unit of analysis. The respective study also had to include both female and male interviewers. After screening we were left with n_s = 169 studies that had to be more closely inspected for eligibility.

To be included as an eligible study, we further refined our selection criteria and inspected whether the studies reported findings on sex-of-interviewer effects on a survey item or item battery (see Table 1, bottom row). Included were only those studies in which the respondent gave a survey response, and wherein at least one numerical outcome was reported to ensure that studies had sufficient focus and analysis on interviewer sex effects. This means that null findings were also included in the review, as long as authors reported statistical results of the statistically insignificant findings. This strategy left us with n_s = 90 eligible studies that included n_d = 100 datasets for the systematic review.

Coding

The coding scheme was developed in an iterative process and included potential moderators and/or mediators to explore what, if anything, influenced the relationship between interviewer sex and respondents’ answers to survey items. We coded the following study characteristics: (1) whether the focus of the study was methodological, (2) the survey design (cross-sectional or longitudinal), (3) the main sampling strategy (probability, non-probability, mixed), (4) whether an interpenetrated design to disentangle spatial and interviewer effects was employed (yes/no), (5) the population (general or special), (6) sample sizes, and (7) the mode of data collection (CAPI/PAPI, CATI, CAWI, mixed). Moreover, we include (8) the total number of interviewers, the number of (9) male and (10) female interviewers, (11) the sex of respondents (male/female/both), (12) whether interviewer and respondent sex matching took place. We also coded (13) the topical scope of the survey and identified eleven wider themes: health, social (attitudes/behavior), family (including household matters), sexuality, crime, gender (including equality issues and feminism), work (including organizational or business matters), politics (including economic issues), demographics, and other. Finally, we also coded (14) the regional reach of the studies (Africa, Australasia, Europe, Latin America, North America). Coding was completed by two independent coders and the intercoder reliability, after resolving discrepancies between coders, was moderate to high (Cronbach’s α = 0.99; Cohen’s κ = 0.72). Any coding discrepancies between the two coders were resolved by discussion.

Analysis

The systematic review begins with a discussion of the overall number of studies examining sex-of-interviewer effects and its variation across time. Next, we quantitatively describe the results of the coding exercise. We focus on the number of studies (n_s) and also include the number of datasets (n_d), given that some studies included more than one dataset. A qualitative review of sex-of-interviewer effects on the survey response is presented subsequently, organized by topical area.

Results: A Systematic Review of Sex-of-Interviewer Effects on the Survey Response

General Observations

First, we discuss the general observations about our sample and the number of studies on sex-of-interviewer effects. Figure 2 shows the number of studies in our sample by year since the initial and ground-breaking work on interviewer effects by Kish (1962). Overall, we found 90 studies (n_s) including 100 datasets (n_d) that related to sex-of-interviewer effects during our field period. The earliest eligible study identified on sex-of-interviewer effects has been published by Colombotos et al. (1968); the newest studies during our field period have been identified as Johann and Mayer (2021), Martin (2021) and Nillesen et al. (2021).

Figure 2.

Number of Studies by Year

Figure 2 further demonstrates that only few studies explored the sex of interviewers until the mid-2000s, when they become more common. Push to web initiatives (e.g., Carpenter & Burton, 2018; Dillman, 2017; Lynn, 2020) may explain the potential pause to sex-of-interviewer studies, as web surveys rarely rely on interviewers. This was surely exacerbated by the Covid-19 pandemic, which paused the number of face-to-face survey data collections. Next, we move on to discussing the overall study characteristics.

Study Characteristics

For each eligible study, their characteristics are summarized in Table 2. The focus of n_s = 73 studies (n_d = 81) was methodological sex-of-interviewer effects, whereas in the remaining studies (n_s = 17; n_d = 19) the focus was substantive wherein sex-of-interviewer effects were secondary, but they were addressed in the analyses. A majority of studies (n_s = 80, n_d = 89) was based on cross-sectional data collection, so only n_s = 10 studies (n_d = 11) implemented a longitudinal study design⁶. We also coded the sampling strategy and found that most studies (n_s = 65, n_d = 70) were based on probability methods, while n_s = 14 studies (n_d = 15) relied on non-probability samples, n_s = 7 studies (n_d = 9) combine probability with non-probability methods, and the remining studies have no information on the sampling strategy (n_s = 6; n_d = 6). Furthermore, we found that only a minority of studies (n_s = 10; n_d = 11) relied on an interpenetrated design that randomly allocated interviewers to primary sampling units, with the vast majority allocating interviewers on the basis of convenience or other rules (n_s = 80; n_d = 89). To our surprise, only n_s = 27 (n_d = 27) studies targeted a sample of the general population, whereas the majority sampled special populations varying from children to patients or academics (n_s = 65; n_d = 73).⁷ The overall sample size varied widely from only 30 to over 50,000 respondents. Regarding the mode of data collection, we found that most studies employed a face-to-face mode, i.e., CAPI or PAPI design (n_s = 54; n_d = 59). Fewer relied on telephone or mobile modes, i.e., CATI (n_s = 27; n_d = 30), an online CAWI mode (n_s = 6; n_d = 6), or a mixed-mode design (n_s = 5; n_d = 5).⁸ It is noteworthy that studies varied widely in the total number of interviewers working on the respective survey project from only 2 interviewers in total up to 605. Similarly wide variations can be observed for the number of male and female interviewers: while the range for female interviewers is 1 to 403, the total number of male interviewers ranged from 1 to 230. Overall, n_s = 20 studies (n_d = 21) did not provide the total number of interviewers; n_s = 26 studies (n_d = 27) did not allow coding the total number of female or male interviewers. A majority of studies included both female and male respondents (n_s = 78; n_d = 86), whereas n_s = 6 studies (n_d = 6) included only female respondents and n_s = 8 studies (n_d = 8) included only males.⁹ Overall, only n_s = 8 studies (n_d = 10) implemented sex-of-interviewer and sex-of-respondent matching for the survey interview; the remaining studies did not apply any matching procedures (n_s = 82; n_d = 90). Finally, in terms of the regional scope, a large number of studies has been conducted in North America (n_s = 44; n_d = 49). Fewer have been conducted in Europe including Norway and Switzerland (n_s = 16, n_d = 17). We also identified studies from Africa (n_s = 18; n_d = 19), Australasia (n_s = 8; n_d = 9), and Latin America (n_s = 4; n_d = 6).

Table 2.

Systematic Review of Sex-of-Interviewer Research

Study ID	Data ID	Author(s)	Year	MethodsFocus	Survey design	Sample	Inter-penetratedDesign	Population	Sample size	Mode	Total NoInt	No maleInt	No femaleInt	SexResp	SexMatching	Region
1	1	Colombotos et al.	1968	Y	Cross-sectional	PR	N	Special	1,713	CAPI/PAPI	31	21	10	Both	N	NORAM
2	2	Cosper	1972	Y	Cross-sectional	PR	N	Special	321	CAPI/PAPI	28	N/A	N/A	Both	N	NORAM
3	3	Landis et al.	1973	Y	Cross-sectional	PR	Y	Special	90	CAPI/PAPI	2	1	1	Both	N	NORAM
4	4	DeLamater	1974	N	Cross-sectional	PR	N	Special	238	CAPI/PAPI	N/A	N/A	N/A	Both	Y	NORAM
5	5	Fhanér & Hane	1974	N	Cross-sectional	PR	N	Special	257	CATI	4	2	2	Both	N	EUR
6	6	Waters	1975	Y	Cross-sectional	NPR	N	Special	48	CAPI/PAPI	4	2	2	Both	N	NORAM
7	7	Johnson & DeLamater	1976	N	Cross-sectional	PR	Y	Special	857	CAPI/PAPI	24	5	19	Both	N	NORAM
7	8	Johnson & DeLamater	1976	N	Cross-sectional	PR	Y	Special	504	CAPI/PAPI	24	5	19	Both	N	NORAM
8	9	Argentino et al.	1977	Y	Cross-sectional	NPR	N	Special	200	CAPI/PAPI	2	1	1	Both	N	NORAM
9	10	Ballou & Del Boca	1980	Y	Cross-sectional	PR	N	Special	1,203	CATI	61	12	49	Both	Y	NORAM
9	11	Ballou & Del Boca	1980	Y	Cross-sectional	PR	N	Special	400	CATI	17	9	8	Both	Y	NORAM
10	12	Galla et al.	1981	Y	Cross-sectional	NPR	N	Special	476	CAPI/PAPI	11	5	6	Both	N	NORAM
11	13	Pearson	1982	N	Cross-sectional	PR	N	Special	305	CATI	20	12	8	Both	N	NORAM
12	14	Grimes & Hansen	1984	N	Cross-sectional	PR	N	Special	216	CATI	N/A	N/A	N/A	Both	N	NORAM
13	15	Groves & Fultz	1985	Y	Longitudinal	PR	Y	General	7,300	CATI	120	40	80	Both	N	NORAM
14	16	Darrow et al.	1986	Y	Cross-sectional	NPR	N	Special	202	CAPI/PAPI	8	5	3	Male	N	NORAM
15	17	Anker et al.	1987	N	Cross-sectional	PR	N	Special	1,621	CAPI/PAPI	13	7	6	Female	N	AASIA
16	18	Chitwood	1988	Y	Cross-sectional	N/A	N	Special	721	CAPI/PAPI	N/A	N/A	N/A	Both	N	NORAM
17	19	Clarke	1989	N	Cross-sectional	NPR	N	General	595	CAPI/PAPI	76	N/A	N/A	Both	N	AASIA
18	20	Pol & Ponzurick	1989	Y	Cross-sectional	PR	N	Special	429	CATI	N/A	N/A	N/A	Both	N	NORAM
19	21	Lueptow et al.	1990	Y	Cross-sectional	PR	N	Special	432	CATI	N/A	N/A	N/A	Both	N	NORAM
20	22	Axinn	1991	Y	Cross-sectional	PR	N	Special	598	CAPI/PAPI	16	N/A	N/A	Female	N	AASIA
21	23	Hutchinson & Wegge	1991	Y	Cross-sectional	PR	N	Special	795	CATI	26	11	15	Both	N	NORAM
22	24	Rubin & Greene	1991	Y	Cross-sectional	NPR	N	Special	247	CAPI/PAPI	N/A	N/A	N/A	Both	N	NORAM
23	25	Lavrakas	1992	N	Cross-sectional	PR	N	Special	400	CATI	13	6	7	Both	N	NORAM
24	26	Walker	1992	Y	Cross-sectional	NPR	N	Special	150	CAPI/PAPI	30	12	18	Both	Y	AASIA
24	27	Walker	1992	Y	Cross-sectional	NPR	N	Special	300	CAPI/PAPI	30	7	23	Both	Y	AASIA
25	28	Blanc & Croft	1992	Y	Cross-sectional	PR	N	Special	4,488	CAPI/PAPI	40	26	14	Female	N	AFR
25	29	Blanc & Croft	1992	Y	Cross-sectional	PR	N	Special	943	CAPI/PAPI	40	26	14	Male	N	AFR
26	30	Johnson & Moore	1993	Y	Cross-sectional	PR	Y	Special	449	CATI	22	7	15	Both	N	NORAM
27	31	Kane & Macaulay	1993	Y	Cross-sectional	PR	Y	General	1,750	CATI	29	14	15	Both	N	NORAM
28	32	Schopper et al.	1993	N	Cross-sectional	PR	Y	Special	1,486	CAPI/PAPI	20	16	4	Female	N	AFR
29	33	Johnson & Parsons	1994	Y	Cross-sectional	PR	N	Special	481	CAPI/PAPI	14	10	4	Both	Y	NORAM
30	34	Becker et al.	1995	Y	Cross-sectional	PR	N	Special	875	CAPI/PAPI	20	12	8	Female	N	AFR
31	35	Catania et al.	1996	Y	Cross-sectional	PR	N	Special	2,030	CATI	40	15	25	Male	Y	NORAM
32	36	Huddy et al.	1997	Y	Cross-sectional	PR	N	Special	303	CATI	20	10	10	Both	N	NORAM
32	37	Huddy et al.	1997	Y	Cross-sectional	PR	N	Special	355	CATI	32	12	20	Both	N	NORAM
33	38	Moum	1998	Y	Cross-sectional	PR	N	General	7,502	Mix	N/A	N/A	N/A	Both	N	EUR
34	39	Pollner	1998	Y	Cross-sectional	PR	N	Special	3,131	CAPI/PAPI	112	62	50	Both	N	NORAM
35	40	Carpenter et al.	1999	Y	Cross-sectional	N/A	N	Special	137	CATI	4	2	2	Female	N	NORAM
36	41	Fendrich et al.	1999	Y	Cross-sectional	NPR	N	Special	3,978	CAPI/PAPI	127	56	71	Male	N	NORAM
37	42	Johnson et al.	2000	Y	Cross-sectional	PR	N	Special	3,714	CATI	27	9	18	Both	N	NORAM
38	43	Dailey & Claus	2001	Y	Cross-sectional	NPR	N	Special	8276	CAPI/PAPI	22	4	18	Both	N	NORAM
39	44	Schräpler	2001	N	Longitudinal	Mix	N	General	45,677	Mix	N/A	N/A	N/A	Both	N	EUR
40	45	Heeb & Gmel	2001	Y	Longitudinal	PR	N	General	2,746	CATI	N/A	N/A	N/A	Both	N	EUR
41	46	McCombie & Anarfi	2002	Y	Cross-sectional	PR	N	Special	1,553	CAPI/PAPI	21	15	6	Both	N	AFR
42	47	Sanbonmatsu	2002	N	Cross-sectional	PR	N	Special	455	CATI	N/A	N/A	N/A	Both	N	NORAM
43	48	Wilson et al.	2002	Y	Cross-sectional	PR	N	Special	1,146	CAPI/PAPI	66	24	42	Both	Y	NORAM
44	49	Nass et al.	2003	N	Cross-sectional	NPR	N	Special	100	CAWI	2	1	1	Both	N	NORAM
45	50	Tourangeau et al.	2003	Y	Cross-sectional	NPR	N	Special	3,047	CAWI	2	1	1	Both	N	NORAM
45	51	Tourangeau et al.	2003	Y	Cross-sectional	PR	N	Special	1,022	CAWI	2	1	1	Both	N	NORAM
46	52	Ahn et al.	2005	Y	Longitudinal	PR	N	Special	30	Mix	2	1	1	Both	Y	AASIA
47	53	Callegaro et al.	2005	Y	Cross-sectional	PR	Y	Special	1,142	CATI	N/A	N/A	N/A	Both	N	NORAM
48	54	Flores-Macias & Lawson	2008	Y	Longitudinal	Mix	N	General	1,600	CAPI/PAPI	75	37	38	Both	N	LATAM
48	55	Flores-Macias & Lawson	2008	Y	Longitudinal	Mix	N	Special	500	CAPI/PAPI	33	16	17	Both	N	LATAM
49	56	Karlan & Zinman	2008	N	Cross-sectional	N/A	N	Special	553	CAPI/PAPI	N/A	N/A	N/A	Both	N	AFR
50	57	Schneider	2008	Y	Cross-sectional	PR	N	Special	1,214	CATI	56	18	38	Both	N	NORAM
51	58	Yang & Yu	2008	Y	Cross-sectional	PR	N	General	1,042	CAPI/PAPI	48	N/A	N/A	Both	N	AASIA
52	59	Fuchs	2009	Y	Cross-sectional	NPR	N	Special	880	CAWI	2	1	1	Both	N	EUR
53	60	Benstead	2010	Y	Cross-sectional	Mix	N	General	800	CAPI/PAPI	20	10	10	Both	N	AFR
54	61	Ackermann & Daley	2010	Y	Cross-sectional	PR	N	General	3,413	CATI	259	118	141	Both	N	NORAM
55	62	Fahrney et al.	2010	Y	Cross-sectional	NPR	N	Special	405	CAWI	2	1	1	Male	N	NORAM
56	63	Wang et al.	2010	Y	Cross-sectional	PR	N	General	2,250	CATI	N/A	N/A	N/A	Both	N	NORAM
57	64	Eisinga et al.	2011	Y	Cross-sectional	PR	N	General	1,212	CAPI/PAPI	98	41	57	Both	N	EUR
58	65	Gong & Aadland	2011	Y	Cross-sectional	PR	N	Special	1,786	CATI	130	35	95	Both	N	NORAM
59	66	Krol et al.	2011	N	Cross-sectional	N/A	N	Special	217	CAPI/PAPI	8	2	6	Male	N	NORAM
60	67	Chun et al.	2011	Y	Cross-sectional	PR	N	Special	474	CAPI/PAPI	14	8	6	Male	N	LATAM
61	68	Sahgal & Horowitz	2011	Y	Cross-sectional	PR	N	General	6,000	CAPI/PAPI	N/A	N/A	N/A	Both	N	AFR
62	69	Bartholomew III	2012	N	Cross-sectional	Mix	N	Special	99	CATI	N/A	N/A	N/A	Both	N	NORAM
62	70	Bartholomew III	2012	N	Cross-sectional	Mix	N	Special	244	CATI	N/A	N/A	N/A	Both	N	NORAM
63	71	Dykema et al.	2012	Y	Longitudinal	N/A	N	Special	776	CAWI	2	1	1	Both	N	NORAM
64	72	Liu & Stainback	2013	Y	Cross-sectional	PR	N	General	3,208	CAPI/PAPI	391	161	230	Both	N	AASIA
65	73	Benstead	2014	Y	Cross-sectional	Mix	N	General	800	CAPI/PAPI	20	10	10	Both	N	AFR
66	74	Benstead	2014	Y	Cross-sectional	Mix	N	General	800	CAPI/PAPI	20	10	10	Both	N	AFR
67	75	Clark	2014	Y	Cross-sectional	PR	N	Special	341	CATI	N/A	N/A	N/A	Male	N	NORAM
68	76	Kappelhof	2014	Y	Cross-sectional	PR	Y	Special	3,287	CAPI/PAPI	N/A	N/A	N/A	Both	N	EUR
69	77	Ngongo et al.	2015	Y	Cross-sectional	PR	Y	Special	805	CAPI/PAPI	3	1	2	Both	N	AFR
70	78	Benstead & Malouche	2015	Y	Cross-sectional	PR	N	General	1,202	CAPI/PAPI	55	28	27	Both	N	AFR
71	79	Houle et al.	2016	Y	Cross-sectional	Mix	N	Special	4,684	CAPI/PAPI	11	5	6	Both	N	AFR
72	80	Lazarus Frankel	2016	Y	Longitudinal	PR	N	General	437	CAPI/PAPI	N/A	N/A	N/A	Both	Y	NORAM
73	81	Oreffice & Quintana-Domeque	2016	N	Cross-sectional	PR	N	Special	1,075	CAPI/PAPI	258	N/A	N/A	Both	N	EUR
74	82	Sheppard	2016	Y	Cross-sectional	PR	N	General	24,170	CATI	397	230	167	Both	N	AASIA
75	83	Lipps & Lutz	2016	Y	Longitudinal	PR	N	General	50,000	CATI	605	202	403	Both	N	EUR
76	84	Crossley et al.	2021	N	Cross-sectional	PR	Y	Special	1,705	CAPI/PAPI	160	99	61	Both	N	EUR
77	85	Grzeszkiewicz-Radulska et al.	2017	Y	Cross-sectional	NPR	N	Special	800	Mix	10	5	5	Both	N	EUR
78	86	Rohm et al.	2017	Y	Cross-sectional	PR	N	Special	5,220	CAPI/PAPI	208	116	92	Both	N	EUR
79	87	Stojmenovska & Steinmetz	2017	Y	Cross-sectional	PR	N	General	32,651	CAPI/PAPI	N/A	N/A	N/A	Both	N	EUR
80	88	Walzenbach et al.	2017	Y	Cross-sectional	PR	N	General	776	CAPI/PAPI	N/A	N/A	N/A	Both	N	EUR
81	89	Vivo et al.	2017	Y	Cross-sectional	PR	N	Special	300	CAPI/PAPI	10	5	5	Both	N	LATAM
81	90	Vivo et al.	2017	Y	Cross-sectional	PR	N	Special	300	CATI	4	2	2	Both	N	LATAM
82	91	Haber et al.	2018	Y	Cross-sectional	PR	N	General	51,423	CAPI/PAPI	201	147	54	Both	N	AFR
83	92	Ne´meth & Luksander	2018	Y	Cross-sectional	N/A	N	General	5,000	CAPI/PAPI	73	N/A	N/A	Both	N	EUR
84	93	Nguyen	2018	Y	Longitudinal	PR	N	Special	2,051	CAPI/PAPI	40	23	17	Both	N	AFR
85	94	Di Maio & Fiala	2020	Y	Cross-sectional	PR	N	Special	6,895	CAPI/PAPI	47	16	31	Both	N	AFR
86	95	Kianersi et al.	2020	Y	Cross-sectional	PR	N	Special	304	CAPI/PAPI	4	2	2	Both	N	LATAM
87	96	Reitmann et al.	2020	Y	Cross-sectional	PR	N	General	2,509	CAPI/PAPI	21	10	11	Both	N	AFR
88	97	Johann & Mayer	2021	Y	Cross-sectional	PR	N	Special	1,976	CAPI/PAPI	183	103	80	Both	N	EUR
88	98	Johann & Mayer	2021	Y	Cross-sectional	PR	N	Special	3,265	CAPI/PAPI	96	42	54	Both	N	EUR
89	99	Martin	2021	Y	Longitudinal	PR	N	General	5,040	Mix	N/A	N/A	N/A	Both	N	AFR
90	100	Nillesen et al.	2021	Y	Cross-sectional	PR	N	General	458	CAPI/PAPI	21	10	11	Both	N	AFR

Note. Y refers to yes, N to no, N/A to not available.

PR refers to probability sampling strategy, NPR to non-probability sampling, Mix to a mixed sampling strategy combining probability and non-probability methods.

CAPI stands for Computer-Assisted-Personal-Interviewing, PAPI for Paper-and-Pencil-Interviewing, CATI for Computer-Assisted-Telephone-Interviewing including mobile phones, CAWI for Computer-Assisted-Web-Interviewing with virtual interviewers, Mix to mixed-mode designs.

No refers to number.

AFR refers to Africa, AASIA includes Australasian countries, EUR refers to Europe including Switzerland and Norway, LATAM to Latin and Middle America, NORAM to North America.

To get a better idea of the topical scope of the reported survey results on sex-of-interviewer effects, we coded up to five topics for each study in our sample. Figure 3 presents an overview of the number of topics (n_t) found across datasets (n_d) and studies (n_d). Note that the number of topics (n_t = 164) exceeds the overall number of datasets (n_d = 100) and studies (n_d = 90), as a single survey data set could include several topics. Overall, we found that the majority of datasets focused on sexuality (n_t = 29), followed by gender (n_t = 28), and health (n_t = 26). Further, we identified that n_t = 20 datasets covered politics, n_t = 19 social issues, and n_t = 18 demographics. Fewer datasets focused on family (n_t = 7), crime (n_t = 6), work (n_t = 6), or other topics (n_t = 5).

Figure 3.

Number of Survey Topics Across Datasets. Note that Datasets/Studies Could Include Multiple Topics, i.e., n_t = 164, n_d = 100, n_s = 90

Qualitative Review of Sex-of-Interviewer Effects on the Survey Response

We qualitatively reviewed the n_s = 90 eligible studies (n_d = 100) regarding sex-of-interviewer effects on the survey response. Acknowledging that the topical scope is relevant to the presence of such effects, we organize our review by topic. Where possible, we identify the over-arching or most typical or systematic outcome for the critical topics,¹⁰ but we do not focus on the magnitude of sex-of-interviewer effects. This would require standardizing, converting and quantitatively summarizing effect sizes as part of a meta-analysis which is beyond the scope of this study. We highlight core findings by topical area in bold.

Sexuality

We would expect that surveys on topics relating to sexuality are more susceptible to sex-of-interviewer effects than other topics, with the modal finding that response patterns are different in same-sex vs opposite-sex pairs. This is especially the case for female respondents, suggesting that homophily (Frankel, 2016), group membership (Schneider, 2008), in-group loyalty (Stojmenovska & Steinmetz, 2017) or self-disclosure theory (Catania et al., 1996, p. 110) play a part in the direction of answers. Generally, women may feel more comfortable disclosing their intimate experiences to other women however in many instances, evidence is too complex to draw underlying conclusions with confidence as sometimes findings present a mix of sex-of-interviewer effects with respect to their direction of bias.

Galla et al. (1981) find that female interviewers elicited more non-traditional responses compared to their male counterparts. Research by Darrow et al. (1986) indicated that female researchers recorded reports of men engaging in homosexual behavior more frequently, while male researchers recorded engagement in oral sex more often. Both studies relied on relatively small sample sizes (e.g., #10, #14). Axinn (1991) suggested that interviewer sex played a role in fertility reports, where female interviewers gathered significantly more progressive responses on contraceptive use and desire for a smaller family size. This study did not report the actual number of male and female interviewers, however (e.g., #2). Becker et al. (1995) find that women refused to answer question about sexual intercourse more frequently when a male interviewer conducted the survey, but also that they were less likely to respond to female interviewers on the use of contraceptives, arguing women might be worried about female gossip. It is worth noting that the number of female and male interviewers is uneven (e.g., #30).

Regarding sexual coercion and abuse, some studies find that survey respondents are more likely to disclose coercion and abuse to male interviewers (Chun et al., 2011; Dailey & Claus, 2001). However, especially the latter study, while relying on large case numbers, recruited respondents employing a non-probability sampling strategy and also included a much larger number of female interviewers (e.g., #38). Other research suggests that female interviewers are more likely to elicit higher reports of condom use (Kianersi et al., 2020; McCombie & Anarfi, 2002). The latter study relies on smaller case numbers and only two male and two female interviewers though (e.g., #86). Ahn et al. (2005) find that men exaggerate their erection function when interviewed by a male interviewer. However, their study relied on small sample size and involved only one male and one female interviewer (e.g., #46). In addition, a study of homosexual men using ACASI indicates that a male voice yielded fewer reports of unprotected sexual intercourse but elicited higher reports of HIV-negative partners (Fahrney et al., 2010). Surveys on sexual practices have produced null findings for potential sex-of-interviewer differences in the past (Blanc & Croft, 1992; Catania et al., 1996; Chitwood, 1988; DeLamater, 1974; Johnson & DeLamater, 1976; Johnson & Moore, 1993). However, some studies relied on non-probability methods, uneven numbers of male and female interviewers, or did not transparently report the number of interviewers or their sex (e.g., #4, #7, #16).

Gender

Given the topical scope, we might expect reporting on topics relating to gender to be generally more prone to sex-of-interviewer effects. The modal finding in the literature indicates that male interviewers have been found to elicit more traditional responses and female interviewers more liberal, progressive or feminist views on gender issues (Ballou & Del Boca, 1980; Benstead, 2010; Flores-Macias & Lawson, 2008; Galla et al., 1981; Huddy et al., 1997; Kane & Macaulay, 1993; Kappelhof, 2014; Lipps & Lutz, 2016; Stojmenovska & Steinmetz, 2017; Wang et al., 2010). Some studies display small or uneven numbers of male and female interviewers (e.g., #9, #75), do not report the number of interviewers nor their sexes (e.g., #68, #79 #56), rely on non-probability samples or mixed sampling designs combining probability with non-probability methods (e.g., #10, #48), or have small sample sizes (e.g., #32). Similar sex-of-interviewer effects are also found regarding survey attitudes towards the gender pay gap (Walzenbach et al., 2017) or the use of gender-exclusive language (Wang et al., 2010). Neither study allowed tracking the number of interviewers by sex. The sex-of-interviewer effect also seems to be present when employing the same male and female virtual interviewers for all respondents (Tourangeau et al., 2003). However, there appears to be variation regarding the wider topic area of gender: some scholars find that male interviewers may trigger more liberal views (Landis et al., 1973) – admittedly, the research relied on a very small sample size and only one male and one female interviewer (e.g., #3). Others suggest that more progressive attitudes are revealed to female interviewers (Walker, 1992), but it appears that also a larger number of women were employed as interviewers (e.g., #24).

Health

We found mixed results about sex-of-interviewer effects on health-related survey items but at large, the modal finding suggests that respondents state better health conditions when interviewed by men (Chun et al., 2011; Lipps & Lutz, 2016; Moum, 1998) and other research indicated that respondents seem more prone to admit to poor health to female interviewers (Krol et al., 2011; Pollner, 1998). We may speculate that respondents may want to look good in front of male interviewers and might feel more comfortable to reveal poor health to women, who might presumably be more caring. It is noteworthy that some studies rely on poor sample quality or small samples (e.g., #59, #60) and an uneven distribution of male and female interviewers (e.g., #59, #75) or lacked information about the interviewers (see e.g., #33).

Looking at substance abuse, research indicates that respondents are more likely to disclose this behavior to male interviewers compared to females. Findings in this light have been found for alcohol abuse (Chun et al., 2011; Cosper, 1972; Johnson & Parsons, 1994) and drug abuse (Johnson & Parsons, 1994). While the above studies claim probability sampling strategies, the case numbers are low with a few hundred participants from special populations (e.g., #2, #60, #26). Moreover, it seems that the number of male and female interviewers is uneven (e.g., #2, #26) or completely unknown (e.g., #60). Studies specifically on drug abuse suggest the opposite effect, i.e., respondents are more likely to report substance abuse to female interviewers (Darrow et al., 1986; Pollner, 1998). The former relies on a non-probability sample, small case numbers and a small number of interviewers unevenly distributed across sexes (e.g., #14). Another two recent studies do not find any sex-of-interviewer differences in reporting substance abuse (Heeb & Gmel, 2001; Johnson et al., 2000). While the former does not report the number and sex distribution of the interviewers, it is noteworthy that the latter employed twice as many female interviewers (e.g., #40, #37).

Research on mental health suggest that respondents provide higher levels of rated subjective anxiety to men (Pollner, 1998; Waters, 1975). Admittedly, the latter study relies on very small case numbers sampled using non-probability methods (e.g., #6). Finally, a study on the willingness of paying higher fees for healthcare products or services (Ngongo et al., 2015) indicates that a female interviewer elicits a greater willingness to pay higher prices. Again, this research relies on one female and one male interviewer only (e.g., #69). Finally, null findings regarding sex-of-interviewer effects are found on restraint and disinhibited eating (Eisinga et al., 2011).

Politics

Overall, we also identify sex-of-interviewer effects on political topics, although the direction of the effect is mixed and does not present clear modal findings. For instance, similarly to the results on gender issues, Ballou and Del Boca (1980) find that female interviewers elicit more progressive responses on a question about women’s movement and research by Kane and Macaulay (1993) indicates that female interviewers trigger more egalitarian and critical responses on inequality. Callegaro et al. (2005) find that white female interviewers trigger more “don’t know” responses, but also that male African-American interviewers elicit a higher willingness to vote for the white, male political candidate than female African-American interviewers. The former finding is also confirmed by research conducted by Schneider (2008). Work by Nguyen (2018) found that female interviewers elicit more frequently respondents’ admission to have taken a loan in the last 12 months but also higher trust in financial institutions; male interviewers seem to trigger higher prevalence of village meetings in the last 12 months. Johann and Mayer’s study (2021) indicates that respondents interviewed by (highly educated) men reveal a higher level of political knowledge. While these studies vary in the mode of data collection, most studies employ high quality sampling strategies, decent sample sizes, as well as even numbers of female and male interviewers. Only one of the studies employed a larger number of female interviewers and a smaller sample in one of the data sets (e.g., #9). Using virtual interviewers, Tourangeau et al. (2003) suggests that respondents report to have turned out in election more often when the virtual interviewer is male. Null effects are found on studies on ideology and political leaders (Benstead & Malouche, 2015; DeLamater, 1974; Lipps & Lutz, 2016; Schneider, 2008) as well as finance (Crossley et al., 2021).

Social Matters

Within the topical area of social matters, sex-of-interviewer effects also matter. The modal finding across several studies indicates that respondents report more progressive social views when the interviewer was female (Ballou & Del Boca, 1980; Dailey & Claus, 2001; Nguyen, 2018; Pearson, 1982; Walzenbach et al., 2017). It also appears that despite the reported random assignment of respondents to interviewers, male interviewers interview more male respondents, fewer respondents over age 60, fewer widowed, more respondents who are working or have higher total family incomes. Benstead (2014) finds that the sex of interviewers and religiosity interact: among religious respondents, female secular interviewers received fewer religious responses compared to men. Furthermore, religious respondents are less likely to say religion is very important when interviewed by a secular female interviewer. Moreover, respondents state a higher willingness to pay for an environmental good when interviewed by female interviewers compared to male interviewers (Gong & Aadland, 2011). Finally, Nass et al. (2003) show that respondents are more likely to reveal sensitive social behavior when prompted by a female voice. However, Tourangeau et al. (2003) suggests that virtual interviewers did not display sex-of-interviewer effects on embarrassing behavior. Admittedly, some studies suffer from poor design characteristics: for instance, unevenly distributed numbers of female and male interviewers (e.g., #13, #9, #38, #58), a lack of information on interviewers (e.g., #80), or small sample sizes (e.g., #11).

Demographics

Sex-of-interviewer effects do not seem to be commonly present in demographic questions, and even when so the modal finding is a null effect suggesting that more factual questions do not elicit any sex-of-interviewer effects. Only, Haber et al. (2018) show that female interviewers increased the odds of reporting asset ownership and household size. Most studies find no substantive sex-of-interviewer effects in respondents’ self-reported educational level (Groves & Fultz, 1985; Oreffice & Quintana-Domeque, 2016), race (Groves & Fultz, 1985), anthropometric measures referring to proportions of the human body (Oreffice & Quintana-Domeque, 2016), but also other socio-demographic attributes including age, occupation, religion, income, or union membership (Hutchinson & Wegge, 1991). Most of these studies rely on quality samples and larger case numbers. Although socio-demographic topics in these manuscripts vary extensively, null findings on sex-of-interviewer effects may not be surprising, given that these are factual questions which may be less prone to misreporting to either sex.

Family

Within the topic area of family and household, we identified few sex-of-interviewer effects, with the modal finding indicating that female interviewers received more pro-family or candid responses related to their household or domestic lives. For instance, female interviewers have been found to elicit higher rates of child deaths in Nigeria (Becker et al., 1995) but also more pro-marriage attitudes than male interviewers (Liu & Stainback, 2013). In line with this finding, research shows that female interviewers are associated with systematically less traditional attitudes towards gender roles in the household amongst Turkish respondents (Kappelhof, 2014). Stojmenovska and Steinmetz (2017) also find small but significant sex-of-interviewer effects on household items. The latter two studies do not provide details about the total number of interviewers or split by sex (e.g., #68, #79). However, null effects of sex-of interviewer effects are prevalent in other studies on home- or family-related content, such as children born or partners’ attitudes (Blanc & Croft, 1992; Kane & Macaulay, 1993; Lueptow et al., 1990).

Crime

The modal finding amongst crime studies indicate that interviewer sex does not influence respondents’ answers on the topic of crime. The only study reporting effects suggests that female interviewers trigger greater concerns about crime (Schräpler, 2001). While the study relies on a large sample, questions need to be raised about the sampling strategy representing a mix of probability and non-probability methods. It also did not specify the total number of interviewers or split by sex.

Work

Looking at work-related topics, we identify some interviewer sex effects in literature although these findings do not present a clear pattern of effects. For example, Anker et al. (1987) find that male interviewers elicit higher labor force participation rates in more specific questions, while female interviewers elicit higher rates for more general questions. Furthermore, research identified some interviewer effects concerning stereotyping occupations on the basis of sex (Clarke, 1989): female interviewers were more likely to reduce sex stereotyping for occupations often associated with men, e.g., university lecturers; male interviewers had a similar effect on stereotyping for jobs often associated with women, e.g., fashion reporters. Axinn (1991) indicated that female interviewers gathered significantly more reports of any wage work, farm work, ownership of farm animals, and money loaning compared to male interviewers. Some effect was identified on work-related items by Kane and Macaulay (1993) where female interviewers triggered more egalitarian responses in favor of gender equality. Martin (2021) finds that female interviewers yielded more negative ratings of journalists in Lebanon and Jordan in a survey on journalistic credibility. However, some studies have little information about the design regarding the total number of interviewers or split by sex (e.g., #17, #20, #89).

Discussion and Conclusions

This article presented a systematic review of sex-of-interviewer effects on the survey response. We posited that while sex-of-interviewer effects have been studied previously (e.g., Catania, Gibson, Chitwood, et al., 1990; Catania, Gibson, Marin, et al., 1990; Davis et al., 2010; Schaeffer et al., 2010; Sudman & Bradburn, 1974; West & Blom, 2017), a systematic review of the findings of potential effects is yet missing. Our article aims to fill this gap by studying publications and grey literature between 1962 and 2022. To our knowledge, this article represents the first systematic review of sex-of-interviewer effects on the survey response. We embedded our research in the Total Survey Error (e.g., Andersen et al., 1979; Groves & Lyberg, 2010; Weisberg, 2009), in which interviewer effects are predominantly associated with measurement error. Our analysis relies on a sample of n_s = 90 studies containing n_d = 100 datasets, which represented a larger sample than we anticipated. We presented our overall observation about the number of studies across time, and systematically discussed the studies’ survey characteristics. We also provided a qualitative review sex-of-interviewer effects by topical scope.

Our findings indicate that the presence and direction of sex-of-interviewer effects may depend on a variety of characteristics. The results lend some confidence that sampling procedures, population types, and sample sizes, and total number of interviewers or for each sex group are relevant characteristics to study and control for when modelling interviewer effects. While the analysis further suggests that the topical scope is important, it may fall too short to argue that some topics are connotated to be more masculine, such as politics and economics, or feminine, such as gender equality and family.

Sensitive questions relating to topical areas such as gender, health or sexuality seem to be more susceptible to sex-of-interviewer effects compared to factual topics such as demographic questions. In addition, future research may wish to investigate the interaction between interviewer and respondent sex. Potential relations between interviewers and respondents have already been observed (e.g., Benstead, 2014; Kane & Macaulay, 1993; Sahgal & Horowitz, 2011; Stojmenovska & Steinmetz, 2017). Researchers may want to re-think the suitability of the mode of data collection, opting for self-administered rather than interviewer-administered modes, or consider sex-matching, if appropriate.

Meta-analysis could help to identify the direction and size of potential effects and to summarize these statistically. However, given the large diversity in statistical methods that prevent converting the statistical findings into comparable effect sizes, this is a challenging task. Especially, changes in attitudes and wider acceptance of topics by the wider society challenge a systematic analysis of key dimensions including susceptibility of topics to sex-of-interviewer effects. This applies to qualitative and quantitative accounts of interviewer effects. It would be desirable that future research finds a way to record and harmonize effect sizes in a coherent way to contribute a full quantitative review of sex-of-interviewer effects. This was beyond the scope of this article. Attention should also be paid to cultural and temporal context of the conducted study as sex-of-interviewer effects may differ in magnitude and direction across different countries and time periods. Moreover, while our research covers a long time period, includes studies around the globe, and reviewed the grey literature, it is limited to English language publications and, as such, lacks evidence from studies reported in other languages. Systematic reviews, such as this one, are important to understand the extent of influences of interviewer sex in survey research and survey data quality in the bigger picture. To be able to achieve this, transparency in the methods reported in individual research papers studying interviewer sex or any other error is vital. We would urge survey methodologists and researchers to continue to transparently document and report the methodological details, so that studies such as ours can be effectively conducted in the future. The above recommendations are summarized in Table 3.

Table 3.

Evidence Based Recommendations

Problem statement	Evidence-based recommendation
Survey is interviewer-administered	• Train interviewers to be aware of interviewer sex effects and to report observations and paradata from fieldwork
	• Employ interpenetrated sampling design to randomly allocate interviewers to sampling units
	• Transparently document and report fieldwork operations in the technical report
Survey topic focuses on gender or related matters (e.g. feminism and gender studies, social issues affecting women, family topics etc.)	• Change survey mode from interviewer-administered to self-administered
	• Randomize interviewer assignment so that interviewer sex is allocated randomly across sampling units
	• Control for interviewer sex in data analysis
Survey focuses on sexuality	• Change survey mode from interviewer-administered to self-administered
Survey focuses on sexuality	• Use same-sex matching to elicit more truthful responses during fieldwork
Survey topic focuses on health	• Change survey mode from interviewer-administered to self-administered
Survey topic focuses on health	• Employ more female vs male interviewers to elicit truthful responses during fieldwork

In sum, our research suggests that it is important to continue conversations about interviewer sex in survey methodology and the wider framework of Total Survey Error. Only if we better understand how sex and other characteristics might influence the survey response, can we start thinking about how to mitigate these effects or how to correct for them post data collection. Our study proposes a few evidence-based recommendation how sex-of-interviewer effects could be addressed in field work.

Footnotes

Acknowledgements

The authors would like to thank Prof. Dr. Rainer Schnell for his valuable guidance in defining the research scope as well as Dr. Vanessa Gash and Dr. Sally Stares for their feedback and suggested improvements to this research.

ORCID iDs

Sonila Dardha

Jessica Daikeler

Kathrin Thomas

Bernd Weiß

Funding

The main author discloses receipt of the following financial support for the research and authorship of this article: This work was supported by the GESIS Eurolab Grant in 2018.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Notes

References

Ahn

T. Y.

Hong

J. H.

Paick

J. S.

Suh

J. K.

Kim

J. J.

Park

N. C.

Park

J. K.

Park

K. L.

Sung

(2005). The influences of interviewer’s gender on the informativeness of self-administered questionnaire for erectile dysfunction. The Journal of Urology, 173(4S), 6. https://doi.org/10.1016/s0022-5347(18)34285-x

Andersen

Kasper

J. D.

Frankel

M. R.

(1979). Total survey error. Jossey-Bass Publishers.

Anker

Khan

M. E.

Gupta

R. B.

(1987). Biases in measuring the labour force: Results of a methods test survey in Uttar Pradesh, India. International Labour Review, 126(2), 151.

Axinn

W. G.

(1991). The influence of interviewer sex on responses to sensitive questions in Nepal. Social Science Research, 20(3), 303–318. https://doi.org/10.1016/0049-089x(91)90009-r

Ballou

Del Boca

F. K.

(1980). Gender interaction effects on survey measures in telephone interviews. In Annual Conference of American Association for Public Opinion. Kings Island, OH, 1980.

Becker

Feyisetan

Makinwa-Adebusoye

(1995). The effect of the sex of interviewers on the quality of data in a Nigerian family planning questionnaire. Studies in Family Planning, 26(4), 233–240.

Benstead

L. J.

(2010). Effects of interviewer gender and Hijab on gender-related survey responses: Findings from a nationally-representative field experiment in Morocco. https://isr.umich.edu/wp-content/uploads/2017/09/lbenstead_paper.pdf (accessed 11 April 2023).

Benstead

L. J.

(2014). Does interviewer religious dress affect survey responses? Evidence from Morocco. Politics and Religion, 7(4), 734–760. https://doi.org/10.1017/s1755048314000455

Benstead

L. J.

Malouche

(2015). Interviewer religiosity and polling in transitional Tunisia. In Annual Conference of the Midwest Political Science Association. Chicago, IL, 2015.

10.

Beullens

Loosveldt

(2016). Interviewer effects in the European social survey. Survey Research Methods, 10(2), 103–118.

11.

Blanc

A. K.

Croft

T. N.

(1992). The effect of the sex of interviewer on responses in fertility surveys: The case of Ghana. In Annual Meeting of the Population Association of America. Denver, CO, 1992.

12.

Callegaro

De Keulenaer

Krosnick

J. A.

Daves

R. P.

(2005). Interviewer effects in a RDD telephone pre-election poll in Minneapolis 2001. An analysis of the effects of interviewer race and gender. In Annual Conference of American Association for Public Opinion, Miami beach, FL, 2005.

13.

Carpenter

Burton

(2018). Adaptive Push-to-Web: Experiments in a household panel study. In Understanding Society Working Paper Series. (5, pp. 1–11). https://www.understandingsociety.ac.uk/sites/default/files/downloads/working-papers/2018-05.pdf (accessed 11 April 2023).

14.

Catania

J. A.

Binson

Canchola

Pollack

L. M.

Hauck

(1996). Effects of interviewer gender, interviewer choice, and item wording on responses to questions concerning sexual behavior. Public Opinion Quarterly, 60(3), 345–375. https://doi.org/10.1086/297758

15.

Catania

J. A.

Gibson

D. R.

Chitwood

D. D.

Coates

T. J.

(1990a). Methodological problems in AIDS behavioral research: Influences on measurement error and participation bias in studies of sexual behavior. Psychological Bulletin, 108(3), 339–362. https://doi.org/10.1037/0033-2909.108.3.339

16.

Catania

J. A.

Gibson

D. R.

Marin

Coates

T. J.

Greenblatt

R. M.

(1990b). Response bias in assessing sexual behaviors relevant to HIV transmission. Evaluation and Program Planning, 13(1), 19–29. https://doi.org/10.1016/0149-7189(90)90005-h

17.

Chitwood

(1988) Effects of interviewer gender on responses to sexual questions by intravenous drug users. Unpublished Raw Data.” Unpublished Manuscript.

18.

Chun

Tavarez

M. I.

Dann

G. E.

Anastario

M. P.

(2011). Interviewer gender and self-reported sexual behavior and mental health among Male military personnel. International Journal of Public Health, 56(2), 225–229. https://doi.org/10.1007/s00038-010-0185-9

19.

Clarke

V. A.

(1989). Occupational sex-typing: A declining problem in Australia. Psychological Reports, 65(3), 1011–1016. https://doi.org/10.2466/pr0.1989.65.3.1011

20.

Colombotos

Elinson

Loewenstein

(1968). Effect of interviewers’ sex on interview responses. Public Health Reports, 83(8), 685–690.

21.

Cosper

(1972). Interviewer effect in a survey of drinking practices. The Sociological Quarterly, 13(2), 228–236. https://doi.org/10.1111/j.1533-8525.1972.tb00806.x

22.

Crossley

T. F.

Schmidt

Tzamourani

Winter

J. K.

(2021). Interviewer effects and the measurement of financial literacy. Journal of the Royal Statistical Society - Series A: Statistics in Society, 184(1), 150–178. https://doi.org/10.1111/rssa.12617

23.

Dailey

R. M.

Claus

R. E.

(2001). The relationship between interviewer characteristics and physical and sexual abuse disclosures among substance users: A multilevel analysis. Journal of Drug Issues, 31(4), 867–888. https://doi.org/10.1177/002204260103100404

24.

Darrow

W. W.

Jaffe

H. W.

Thomas

P. A.

Haverkos

H. W.

Rogers

M. F.

Guinan

M. E.

Auerbach

M. D.

Spira

Curran

J. W.

(1986). Sex of interviewer, place of interview, and responses of homosexual men to sensitive questions. Archives of Sexual Behavior, 15(1), 79–88. https://doi.org/10.1007/BF01542306

25.

Davis

R. E.

Couper

M. P.

Janz

N. K.

Caldwell

C. H.

Resnicow

(2010). Interviewer effects in public health surveys. Health Education Research, 25(1), 14–26. https://doi.org/10.1093/her/cyp046

26.

DeLamater

J. D.

(1974). Methodological issues in the study of premarital sexuality. Sociological Methods & Research, 3(1), 30–61. https://doi.org/10.1177/004912417400300102

27.

Dillman

D. A.

(2017). The promise and challenge of pushing respondents to the web in mixed-mode surveys. Survey Methodology, 43(1), 3–31.

28.

Durrant

G. B.

D’Arrigo

(2014). Doorstep interactions and interviewer effects on the process leading to cooperation or refusal. Sociological Methods & Research, 43(3), 490–518. https://doi.org/10.1177/0049124114521148

29.

Eisinga

Grotenhuis

Larsen

J. K.

Pelzer

Strien

(2011). BMI of interviewer effects. International Journal of Public Opinion Research, 23(4), 530–543. https://doi.org/10.1093/ijpor/edr026

30.

Fahrney

Uhrig

Kuo

T. M.

(2010). Gender-of-Voice effects in an ACASI study of same-sex behavior. RTI Press.

31.

Fellegi

I. P.

(1964). Response variance and its estimation. Journal of the American Statistical Association, 59(308), 1016–1041. https://doi.org/10.1080/01621459.1964.10480747

32.

Fendrich

Johnson

Shaligram

Wislar

J. S.

(1999). The impact of interviewer characteristics on drug use reporting by Male juvenile arrestees. Journal of Drug Issues, 29(1), 37–58. https://doi.org/10.1177/002204269902900103

33.

Fhanér

Hane

(1974). Seat belts: Contextual factors and bias of reported use. Journal of Safety Research, 6(4), 166–170.

34.

Flores-Macias

Lawson

(2008). Effects of interviewer gender on survey responses: Findings from a household survey in Mexico. International Journal of Public Opinion Research, 20(1), 100–110. https://doi.org/10.1093/ijpor/edn007

35.

Fowler

F. J.

Mangione

T. W.

(1990). Standardized survey interviewing: Minimizing interviewer-related error. Sage Publications.

36.

Frankel

L. L.

(2016). The opposing effects of matching interviewer and respondent characteristics in longitudinal surveys. Working paper.

37.

Galla

J. P.

Frisone

J. D.

Jeffrey

L. R.

Gaer

E. P.

(1981). Effect of experimenter’s gender on responses to a sex-role attitude questionnaire. Psychological Reports, 49(3), 935–940. https://doi.org/10.2466/pr0.1981.49.3.935

38.

Goffman

(1979). Gender advertisements. The Macmillan Press Ltd.

39.

Gong

Aadland

(2011). Interview effects in an environmental valuation telephone survey. Environmental and Resource Economics, 49(1), 47–64. https://doi.org/10.1007/s10640-010-9423-0

40.

Groves

R. M.

(1989). Survey costs and survey errors. Wiley.

41.

Groves

R. M.

(2004). Interviewer falsification in survey research: Current best methods for prevention, detection, and repair of its effects. Survey Research, 35(1), 1–5.

42.

Groves

R. M.

Fowler

F. J.

Couper

M. P.

Lepkowski

J. M.

Singer

Tourangeau

(2004). Survey methodology. Wiley.

43.

Groves

R. M.

Fultz

N. H.

(1985). Gender effects among telephone interviewers in a survey of economic attitudes. Sociological Methods & Research, 14(1), 31–52. https://doi.org/10.1177/0049124185014001002

44.

Groves

R. M.

Lyberg

(2010). Total survey error: Past, present, and future. Public Opinion Quarterly, 74(5), 849–879. https://doi.org/10.1093/poq/nfq065

45.

Haber

Robyn

P. J.

Hamadou

Yama

Hien

Louvouezo

Fink

(2018). Surveyor gender modifies average survey responses: Evidence from household surveys in four Sub-Saharan African countries. https://arxiv.org/ftp/arxiv/papers/1810/1810.01981.pdf. (accessed 11 April 2023).

46.

Heeb

Gmel

(2001). Interviewers’ and Respondents’ effects on self-reported alcohol consumption in a Swiss health survey. Journal of Studies on Alcohol, 62(4), 434–442. https://doi.org/10.15288/jsa.2001.62.434

47.

Hox

J. J.

Leeuw

E. D.

Kreft

I. G. G.

(2004). The effect of interviewer and respondent characteristics on the quality of survey data: A multilevel model. In Biemer

P. P.

Lyberg

L. E.

Mathiowetz

N. A.

Sudman

(Eds.), Measurement errors in surveys (pp. 439–461). Wiley & Sons.

48.

Huddy

Billig

Bracciodieta

Hoeffler

Moynihan

P. J.

Pugliani

(1997). The effect of interviewer gender on the survey response. Political Behavior, 19(3), 197–220. https://doi.org/10.1023/a:1024882714254

49.

Hutchinson

K. L.

Wegge

D. G.

(1991). The effects of interviewer gender upon response in telephone survey research. Journal of Social Behavior & Personality, 6(3), 573–584.

50.

Hyman

H. H.

(1954). Interviewing in social research. The University of Chicago Press.

51.

Johann

Mayer

S. J.

(2021). Do interviewers affect measures of factual political knowledge? Evidence from Austria and Germany. International Journal of Public Opinion Research, 33(4), 998–1011. https://doi.org/10.1093/ijpor/edab002

52.

Johnson

Fendrich

Shaligram

Garcy

Gillespie

(2000). An evaluation of the effects of interviewer characteristics in an RDD telephone survey of drug use. Journal of Drug Issues, 30(1), 77–101. https://doi.org/10.1177/002204260003000105

53.

Johnson

Moore

R. W.

(1993). Gender interactions between interviewer and survey respondents: Issues of pornography and community standards. Sex Roles, 28(5–6), 243–261. https://doi.org/10.1007/bf00289884

54.

Johnson

Parsons

J. A.

(1994). Interviewer effects on self-reported substance use among homeless persons. Addictive Behaviors, 19(1), 83–93. https://doi.org/10.1016/0306-4603(94)90054-x

55.

Johnson

W. T.

DeLamater

J. D.

(1976). Response effects in sex surveys. Public Opinion Quarterly, 40(2), 165–181. https://doi.org/10.1086/268285

56.

Kane

E. W.

Macaulay

L. J.

(1993). Interviewer gender and gender attitudes. Public Opinion Quarterly, 57(1), 1–28. https://doi.org/10.1086/269352

57.

Kappelhof

J. W.

(2014). The impact of method bias on the cross-cultural comparability in face-to-face surveys among ethnic minorities. Methods, Data, Analyses, 8(1), 79–118.

58.

Kianersi

Luetke

Jules

Rosenberg

(2020). The association between interviewer gender and responses to sensitive survey questions in a sample of Haitian women. International Journal of Social Research Methodology, 23(2), 229–239. https://doi.org/10.1080/13645579.2019.1661248

59.

Kish

(1962). Studies of interviewer variance for attitudinal variables. Journal of the American Statistical Association, 57(297), 92–115. https://doi.org/10.1080/01621459.1962.10482153

60.

Krol

A. L.

Mrazik

Naidu

Brooks

B. L.

Iverson

G. L.

(2011). Assessment of symptoms in a concussion management programme: Method influences outcome. Brain Injury, 25(13–14), 1300–1305. https://doi.org/10.3109/02699052.2011.624571

61.

Krumpal

(2013). Determinants of social desirability bias in sensitive surveys: A literature review. Quality and Quantity, 47(4), 2025–2047. https://doi.org/10.1007/s11135-011-9640-9

62.

Landis

Sullivan

Sheley

(1973). Feminist attitudes as related to sex of the interviewer. Pacific Sociological Review, 16(3), 305–314. https://doi.org/10.2307/1388489

63.

Lipps

Lutz

(2016). Gender of interviewer effects in a multitopic centralized CATI panel survey. Methods, Data, Analyses, 11(1), 67–86.

64.

Liu

Stainback

(2013). Interviewer gender effects on survey responses to marriage-related questions. Public Opinion Quarterly, 77(3), 606–618. https://doi.org/10.1093/poq/nft019

65.

Loosveldt

(2008). Face-to-Face interviews. In Leeuw

Hox

Dillman

D. A.

(Eds.), International handbook of survey methodology (pp. 201–220). Taylor & Francis.

66.

Lueptow

L. B.

Moser

S. L.

Pendleton

B. F.

(1990). Gender and response effects in telephone interviews about gender characteristics. Sex Roles, 22(1–2), 29–42. https://doi.org/10.1007/bf00288152

67.

Lynn

(2020). Evaluating push-to-web methodology for mixed-mode surveys using address-based samples. Survey Research Methods, 14(1), 19–30.

68.

Mangione

T. W.

Fowler

F. J.

Louis

T. A.

(1992). Question characteristics and interviewer effects. Journal of Official Statistics, 8(3), 293–307.

69.

Martin

J. D.

(2021). It depends on who’s asking: Interviewer gender effects on credibility ratings of Male and female journalists in six Arab countries. International Journal of Public Opinion Research, 33(1), 18–37. https://doi.org/10.1093/ijpor/edz053

70.

McCombie

S. C.

Anarfi

J. K.

(2002). The influence of sex of interviewer on the results of an AIDS survey in Ghana. Human Organization, 61(1), 51–57. https://doi.org/10.17730/humo.61.1.em6l865y3v9y7y2l

71.

Moher

Liberati

Tetzlaff

Altman

D. G.

Prisma Group . (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. Journal of Clinical Epidemiology, 6(7), 1–6. https://doi.org/10.1016/j.jclinepi.2009.06.005

72.

Moum

(1998). Mode of administration and interviewer effects in self-reported symptoms of anxiety and depression. Social Indicators Research, 45(1–3), 279–318. https://doi.org/10.1023/a:1006958100504

73.

Nass

Robles

Heenan

Bienstock

Treinen

(2003). Speech-based disclosure systems: Effects of modality, gender of prompt, and gender of user. International Journal of Speech Technology, 6(2), 113–121. https://doi.org/10.1023/a:1022378312670

74.

Ngongo

C. J.

Frick

K. D.

Hightower

A. W.

Mathingau

F. A.

Burke

Breiman

R. F.

(2015). The perils of straying from protocol: Sampling bias and interviewer effects. PLoS One, 10(2), 1–11. https://doi.org/10.1371/journal.pone.0118025

75.

Nguyen

P. L.

(2018). Evidence for gender of interviewer effect from a face-to-face survey in Zambia. University of Nebraska. https://digitalcommons.unl.edu/cgi/viewcontent.cgi?article=1037&context=sociw (accessed 11 April 2023).

76.

Nillesen

Grimm

Goedhuys

Reitmann

A. K.

Meysonnat

(2021). On the malleability of gender attitudes: Evidence from implicit and explicit measures in Tunisia. World Development, 138(February), 1–14. https://doi.org/10.1016/j.worlddev.2020.105263

77.

Oreffice

Quintana-Domeque

(2016). Beauty, body size and wages: Evidence from a unique data set. Economics and Human Biology, 22(September), 24–34. https://doi.org/10.1016/j.ehb.2016.01.003

78.

Pearson

J. C.

(1982). The role of gender in source credibility. Women's Caucus, Speech Communication Association Convention, Louisville, KY. https://files.eric.ed.gov/fulltext/ED226390.pdf (accessed 11 April 2023).

79.

Phillips

D. L.

Clancy

K. J.

(1972). Some effects of ‘Social Desirability’ in survey studies. American Journal of Sociology, 77(5), 921–940. https://doi.org/10.1086/225231

80.

Pol

L. G.

Ponzurick

T. G.

(1989). Gender of interviewer/gender of respondent bias in telephone surveys. Applied Marketing Research, 29(2), 9–13.

81.

Pollner

(1998). The effects of interviewer gender in mental health interviews. The Journal of Nervous and Mental Disease, 186(6), 369–373. https://doi.org/10.1097/00005053-199806000-00008

82.

Rubin

D. L.

Greene

K. L.

(1991). Effects of biological and psychological gender, age cohort, and interviewer gender on attitudes toward gender-inclusive/exclusive language. Sex Roles, 24(7–8), 391–412. https://doi.org/10.1007/bf00289330

83.

Sahgal

Horowitz

J. M.

(2011). Interviewer gender effects in international surveys: An analysis of Muslim publics in six countries. In: Annual Conference of American Association for Public Opinion. Phoenix, AZ, 2011.

84.

Schaeffer

N. C.

Dykema

Maynard

D. W.

(2010). Interviewers and interviewing. In Marsden

P. V.

Wright

J. D.

(Eds.), Handbook of Survey Research (pp. 437–471).

85.

Schneider

(2008). Measurement in Surveys and Elections: Interviewer Effects in Election Surveys, Name Order on Election Ballots, Customer Satisfaction Surveys. Doctoral dissertation. Stanford University.

86.

Schnell

Kreuter

(2005). Separating interviewer and sampling-point effects. Journal of Official Statistics, 21(3), 389–410.

87.

Schräpler

(2001). Respondent behavior in panel studies. Respondent behavior in panel studies: A case study of the German socio-economic panel. DIW Discussion Papers, 244, 1–27.

88.

Stojmenovska

Steinmetz

(2017). Survey response bias and gender ideology: A multi-level study on cross-country differences in gender-of-interviewer effects. In Annual Conference of the European Survey Research Association. Lisbon, Portugal, 2017.

89.

Sturgis

(2014). Interviewers and Survey error: Overview and introduction. In Scherpenzeel

Jann

Sturgis

Lynn

Engel

(Eds.), Improving survey methods. Lessons from recent research (pp. 91–93). Routledge.

90.

Sudman

Bradburn

N. M.

(1974). Response effects in surveys: A review and synthesis. Aldine.

91.

Sullivan

(2020). Sex and the census: Why surveys should not conflate sex and gender identity. International Journal of Social Research Methodology, 23(5), 517–524. https://doi.org/10.1080/13645579.2020.1768346

92.

Tourangeau

Couper

M. P.

Steiger

D. M.

(2003). Humanizing self-administered surveys: Experiments on social presence in web and IVR surveys. Computers in Human Behavior, 19(1), 1–24. https://doi.org/10.1016/s0747-5632(02)00032-8

93.

Tourangeau

Rasinski

K. A.

(1988). Cognitive processes underlying context effects in attitude measurement. Psychological Bulletin, 103(3), 299–314. https://doi.org/10.1037/0033-2909.103.3.299

94.

Tourangeau

Rips

L. J.

Rasinski

K. A.

(2000). The psychology of survey response. Cambridge University Press.

95.

Walker

(1992). Effects of interviewer’s sex on responses to the attitudes toward women scale. The Journal of Social Psychology, 132(5), 675–677. https://doi.org/10.1080/00224545.1992.9713906

96.

Walzenbach

Auspurg

Hinz

(2017). Interviewer effects in factorial survey experiments. In Annual Conference of the European Survey Research Association, Lisbon, Portugal, 2012.

97.

Wang

Kiley

Morin

(2010). Gender of interviewer: How does it influence responses in a gender-focused survey? In Annual Conference of the American Association for Public Opinion Research. Chicago, IL, 2010.

98.

Waters

T. J.

(1975). Further comparison of video tape and face-to-face interviewing. Perceptual and Motor Skills, 41(3), 743–746. https://doi.org/10.2466/pms.1975.41.3.743

99.

Weisberg

H. F.

(2009). The total survey error approach: A guide to the new science of survey research. University of Chicago Press.

100.

West

B. T.

Blom

A. G.

(2017). Explaining interviewer effects: A research synthesis. Journal of Survey Statistics and Methodology, 5(2), 175–211.

101.

West

B. T.

Kreuter

(2015). A practical technique for improving the accuracy of interviewer observations of respondent characteristics. Field Methods, 27(2), 144–162. https://doi.org/10.1177/1525822x14549429

102.

Yang

M. L.

R. R.

(2008). The interviewer effect when there is an education gap with the respondent: Evidence from a survey on biotechnology in Taiwan. Social Science Research, 37(4), 1321–1331. https://doi.org/10.1016/j.ssresearch.2008.05.008

Let’s Talk About Sex: A Systematic Review of Sex-of-Interviewer Effects on the Survey Response

Abstract

Keywords

Introduction

Interviewer Effects and the TSE Framework

Study Design

Selection Strategy

Coding

Analysis

Results: A Systematic Review of Sex-of-Interviewer Effects on the Survey Response

General Observations

Study Characteristics

Qualitative Review of Sex-of-Interviewer Effects on the Survey Response

Sexuality

Gender

Health

Politics

Social Matters

Demographics

Family

Crime

Work

Other Topics

Discussion and Conclusions

Footnotes

Acknowledgements

ORCID iDs

Funding

Declaration of Conflicting Interests

Notes

References