Landline versus Cell Phone Surveys – Interviewers' Experience

Abstract

French

The aim of this paper is to outline the results of the study that was carried out among CATI (Computer Assisted Telephone Interview) interviewers from October 2009 to August 2010. 12 major Polish research organizations as well as two companies in Norway and Iceland participated in the research. The research was based on a standardized self-completion questionnaire (in total 942 interviewers were surveyed) and in-depth interviews (IDI) which were conducted with 49 experienced CATI interviewers. This paper focuses on specific results of the qualitative part of the project; it investigates difficult situations encountered by the interviewers while conducting telephone surveys. During in-depth interviews, the interviewers were encouraged to describe the differences between “mobile” and “landline” respondents, especially in terms of the persuasiveness of the introductory speech and communication interferences during question-asking and answer-recording stage. The interviewers were also asked to elaborate on the relationship between the probability of encountering difficulties while talking to the respondents on the mobile / landline phone and different features, such as the B2B/B2C (« business to business »/« business to consumers ») character of the project, socio-demographic characteristics of the respondent, sampling procedures (RDD versus list-assisted samples), etc.

Keywords

CATI Enquêteurs Enquêtes téléphoniques Téléphonie fixe Téléphonie mobile Interviewers Telephone Surveys Landline Telephony Mobile Telephony

Introduction

Contrary to face-to-face interviews, in CATI the interviewer is not physically present while administrating the survey. Therefore, communication between the interviewer and the respondent lacks the non-verbal component. This lack of non-verbal communication influences both the quality of respondents’ answers, as well as the interviewer-respondent interaction (De Leeuw, 1992; 2005).

On the one hand, absence of visual contact between the interviewer and the respondent has a positive impact on the data obtained, as it reduces the interviewer burden. According to the results of research, CATI interviewers tend to influence the respondents’ answers slightly less often than face-to-face interviewers (e.g., Tucker, 1983; Groves and Magilavy, 1986).

On the other hand, introductory speeches in telephone interviews are likely to be less persuasive and less effective. Additionally, CATI respondents can easily hang up, even while in the question-asking and answer-recording stage of the interview (Couper and Groves, 2002). Moreover, in telephone conversations the interviewer is not able to monitor the conditions in which the respondent answers the questions. It is typical of CATI (especially on mobile phones) that the respondent is engaged in other activities at the time of the interview, for instance, doing household chores, participating in meetings, driving a car, etc. (Holbrook et al., 2003).

Another disadvantage of telephone interviews is the fact that the pace of such conversation is faster and the respondents are more likely to avoid silences than in face-to-face contact. These characteristics have an impact on the answers obtained. One of the consequences is satisficing (Krosnick, 1999). Satisficing refers to a respondent’s tendency to reduce the burden of responding to the survey question. As a result of satisficing, respondents will often fail to differentiate between items that have the same response options (Krosnick, 1999; Lynn and Kaminska, 2012), exhibit primacy and recency effects – the tendency to select one of the first or last response options – or give responses like “hard to say” if these options are presented explicitly (Jordan et al., 1980; Groves, 1979). Moreover, several studies have demonstrated that in telephone interviews respondents’ answers to open-ended questions are likely to be shorter and less complex (Sykes and Collins, 1988).

As far as telephone interviews’ susceptibility to social desirability bias is concerned, the results of research seem to be inconsistent. In some comparisons (de Leeuw and van der Zouwen, 1988; Holbrook et al., 2003) small effect for social desirability was found in favor of face-to-face surveys. In others (Kormendi, 1988; Sykes and Collins, 1988) the item “social desirability” did not show statistically significant differences.

Obviously, telephone interviewers are much more constrained than face-to-face interviewers as far as using visual aids is concerned. Lack of visual materials is usually compensated by the implementation of special techniques while designing the interview script. Such techniques include using semantic scales instead of numeric ones, as well as unfolding questions based on semantic scales (Groves, 1979; Dillman, 1978; Malhotra et al., 2009). Unfolding means that a general question is asked (e.g., if the respondent is satisfied with something) and depending on the respondent’s answer, the interviewer asks a more specific question (e.g., if the respondent is very satisfied or somewhat satisfied). Other recommended techniques include using fewer answer categories and shortening the answer categories, as well as formulating pre-categorised questions (Frey, 1989; de Leeuw, 2002).

In recent years mobile phone technology has posed a serious challenge to the survey industry. Nowadays, a considerable number of households cannot be reached by landline surveys as they rely predominantly or exclusively on mobile phones (Blyth, 2008; Keeter et al., 2007; Mohorko et al., 2013). This situation differs significantly by country; for instance, in the United States mobile only households constitute 38 percent of overall households (Blumberg and Luke, 2013); in Sweden – 2 percent; in Czech Republic – 78 percent (European Commission, 2012). In general, for most telephone surveys it is necessary to implement sampling frames, using landline as well as mobile phone numbers.

In a growing body of the literature, the issue concerning mixing landline and mobile phone numbers in survey research is widely discussed. However, not all aspects of this topic are equally covered. While sampling, weighting and nonresponse are well studied (e.g., Brick and Lepkowski, 2008; Lee and Valliant, 2008; Keeter et al., 2007), few studies have examined measurement errors in dual-frame telephone surveys.

It can be expected that respondents on mobile phones are more likely to be in public places, in the presence of third parties, and to multitask at the time of the interview. Also, due to poor network coverage or interference, the quality of the line can be unsatisfactory. All these factors can influence the way the respondent understands the survey questions and responds to them (Lynn and Kaminska, 2012; AAPOR, 2010). On the other hand, people talking on mobile phones are less physically constrained than those using landline phones; usually they can easily change their location. For instance, they can move to a place with better network coverage. Additionally, the relationship between the respondent and third parties is different for mobile interviews than for landline CATI surveys – people who might listen in on the interview are usually strangers (Haeder, 2012).

In general, as far as the accuracy of the respondents’ answers are concerned, empirical studies have indicated that there is no significant difference between respondents’ reactions in telephone surveys using mobile and landline numbers (Brick et al., 2006; Brick et al., 2007; Kuehne and Haeder, 2012; Haeder, 2012). The mode itself (CATI using landline or mobile telephony) is not the determinant of the response behavior, but there are other variables potentially associated with the mode. For instance, Kuehne and Haeder (2012) have discovered that socio-demographic characteristics have an influence on data quality. Education is especially useful for explaining differences in the quality of answers – the higher the level of education, the higher the data quality. In other research (also concerning telephone surveys but not comparing mobile to landline telephony) it has been indicated that poor data quality may also be associated with respondent age – the older the respondent, the worse the data quality (Loosveldt, 1997) – and the length of the interview – the longer the conversation, the worse the data quality (Holbrook et al., 2003; Frey, 1989).

Interviewers’ Opinions as Indicators of Data Quality

In this article we examine CATI interviewers’ opinions of their work, specifically the difficulties they encounter while administering the survey, and the solutions they implement in order to overcome these problems. We consider the interviewers’ opinions of an interview to be reliable indicators of the quality of the conversation. However, it must be noted that in a growing body of the literature few research results refer to this problem.

Tarnai and Paxson (2005) summarized the results of nearly 25 thousand interviews using 12 different telephone surveys. The interviewers were asked to rate the quality of the interview, and to indicate why an interview may not have gone as well as expected. Only 10 percent of interviews were deemed as less than perfect. Among the top problems most frequently indicated by interviewers were: “respondent did not understand meaning of some questions” (21 percent); “hearing problems” (11 percent); “interruptions and distractions” (9 percent); “interview not in respondent’s native language” (9 percent); and “infirm” (7 percent). A correlation between rating quality and the average number of missing items was noted insofar as the fewer the number of missing items, the better the interviewer’s rating of a particular survey. The authors claim that in many cases interviewers are able to identify the source of difficulties encountered by the respondents. For instance, a large number of respondents encountering problems in understanding questions might suggest that some changes in the CATI script are needed, or that the interviews should be conducted in a more conversational manner (see Conrad and Schober, 2000; Schober and Conrad, 1997).

Fazzi et al. (2009) described the results of their qualitative research, which was carried out among 26 telephone interviewers working in various Italian survey agencies. A series of in-depth interviews were conducted, and the interviewers were asked to discuss their experiences associated with conducting interviews. In particular, interviewers were asked how often and in what context they were likely to infringe the standardization rule. The interviewers referred to many situations in which they would not adhere to a standardized script when conducting interviews. They reported that they departed from a standardized set of rules in order to make the interview less formal and more pleasant for the respondent. Furthermore, they also indicated that they destandardized the interview protocol to minimize the risk of having the respondent terminate the interview. The authors claim that the regulations often outlined in interviewers’ manuals do not correspond to the reality of fieldwork. In their opinion, CATI interviewers should be formally trained in how to handle difficult situations that may occur during telephone interviews.

In this paper we will present the results of a study conducted among CATI interviewers. The two main questions explored include: 1) what kind of strategies are used by the interviewers when they are in contact with problem respondents who behave contrary to interviewers’ and researchers’ expectations; and 2) to what extent is the character of problem situations encountered by the interviewers associated with the type of telephony used in the survey (mobile versus landline).

Methodology

The study was conducted between November 2009 and August 2010. 12 major Polish commercial survey organizations participated in this project: 4P research mix; ASM Centrum Badan i Analiz Rynku; ARC Rynek i Opinia; Expert-Monitor (at present: Kantar Media); GfK Polonia; IMAS International; IPSOS; Grupa IQS, MillwardBrown SMG/KRC (at present: MillwardBrown); PBS DGA (at present: PBS); Pentor Research International (at present: TNS Polska); and TNS OBOP (at present: TNS Polska). Each of these companies has CATI facilities, and each carries out telephone interviews on a regular basis. The criterion by which organizations were selected to participate was possession of a certificate in the CATI category issued by the Interviewing Quality Control Programme.¹ In 2009, a total of 16 firms were certified; however, six of them did not participate in the study – either they refused to participate, or their studio had been closed down.

A total of 846 interviewers took part in this study. Response rates fluctuated between 20 percent and 100 percent (response rate was defined as the number of completed questionnaires divided by the number of interviewers who were at work at least once within one month before the research was conducted). Two methods of data collection were used. Across 10 agencies, interviewers were asked to complete paper questionnaires. In two other companies we used an electronic questionnaire (CASI or CAWI). Additionally, we conducted 32 in-depth interviews with CATI interviewers. Interviewers were selected if they had at least one year of work experience, based on sex, age, and level of education to ensure that the sample of participants reflected the demographics of the interviewers across all the participating companies.

Similar research was conducted in two international agencies – Capacent Gallup in Iceland and Statistics Norway (Statistik sentralbyra). A total of 96 Norwegian and Icelandic CATI interviewers participated in the quantitative part of the project and completed paper questionnaires. In-depth interviews were conducted with 17 experienced interviewers. The fieldwork procedures used in both companies reflect those implemented in Poland.

Below we will present the results of the qualitative part of this project. The IDI scenario consisted of different issues, mainly referring to problem situations CATI interviewers encounter while talking to respondents. The interviewers were encouraged to describe strategies they use in order to deal with problem respondents. Moreover, they were also asked to elaborate on the relationships between the character of encountered difficulties and the landline/mobile type of the telephony.

As the core part of the project was conducted in Poland, the presented results will refer to the Polish subsample of interviewers. At the end of this paper, we will point out the main differences between Polish and Norwegian/Icelandic interviewers’ opinions.

Problem Respondents – Typology

We singled out the following types of problem respondents:

Reluctant Respondents – refusing to participate in the interview;

Irritated Respondents – complaining about the interview;

Respondents Having Difficulties in understanding the questions;

Talkative Respondents – making irrelevant comments about the questions, engaging in digressions, etc.

Problem Respondents – Characteristics

Reluctant Respondents

A majority of interviewers claimed that while talking to a reluctant respondent they calculate the potential profitability of persuading the person to take part in the interview. If the respondent is easily available (i.e., there are a lot of people that fit the specified and required characteristics in the population) and/or the sampling frame is rich in telephone numbers, the interviewers usually give up persuading, say goodbye and call another number. If the sampling frame is limited and/or the recruitment criteria are highly specialized, most interviewers attempt to encourage reluctant respondents to participate.

If it’s an ordinary survey, I don’t respect respondents very much. If they don’t agree from the first, talking to them is a waste of time. If it’s a research with limited frame, I have, let’s say, 10 numbers and I have to conduct as many interviews as it’s possible, it’s much more difficult, here I must try. (Woman, 23 years old, CATI interviewer for 2 years)

In general, I don’t exaggerate persuading. I’m always afraid of the fact that I start the interview and the respondent refuses in the middle of the conversation. Why should I take a risk? Another respondent can have no trouble agreeing. (Woman, 25 years old, CATI interviewer for 1.5 years)

Sometimes the decision of whether a respondent should be persuaded to complete the interview or not is based on the interviewers’ moods. For instance, if the interview feels like they want to have a discussion engage in an argument with someone, or they wish to keep themselves amused, they would prefer having a conversation with the respondents. In such cases, the interviewers’ behavior seems to be associated with the steadfastness of the respondent’s refusal. Specifically, if participants are particularly adamant in refusing to complete the survey, most interviewers will end such contact and terminate the survey. In addition, interviewers may also refrain from persuading the respondent if the interviewer senses the respondent may experience difficulties, cognitive or otherwise, while completing the survey.

If I go to work for 8 hours, during first 4 hours I persuade, because I like doing it, besides time flies when I talk to people instead of hearing the tone (…). Afterwards, when I’m tired, if somebody refuses, I say, “thank you very much.” (Woman, 24 years old, CATI interviewer for 1 year)

It depends on my humor. If I’m in a good mood, I like bickering with stubborn respondents, and sometimes I manage to persuade someone (…). But when I’m in a bad mood, I don’t feel like doing it and I give up. (Man, 25 years old, CATI interviewer for 1 year)

There are two situations when I know in advance that I won’t win anything: strong voice saying “no” and an old woman who doesn’t understand anything, and – even when she agrees to participate – she won’t answer any question. (Man, 25 years old, CATI interviewer for 2 years)

In general it can be assumed that the level of persuasion in the introductory speech of CATI interviews is relatively poor. Interviewers tend to conduct as many interviews as possible within a limited timeframe. As some interviewers report, many refusals could be converted to successful responses if interviewers put forth more effort in their introductory speeches.

Practice proves that persons who have just refused can agree later and even be ideal respondents. Sometimes we make calls in inconvenient moments and that is why the respondents say “no.” I often schedule such persons for – let’s say – 3 hours and make a note “unkind guy.” Afterwards it turns out that someone was bawling by his boss when I was calling him, and now he’s at home eating supper, he’s bored (…). Once we had very limited frame and we finished all numbers after one day. Next day the supervisor gave us the same numbers, saying that the frame is new. Suddenly, 1/3 of previously rejected respondents agreed to participate. (Man, 25 years old, CATI interviewer for 1 year)

Irritated Respondents

Irritated respondents that complain about the interview are another type of problem respondent. The length of the interview most often causes this irritation, as the respondents indicate that the interviews in which they participate should be shorter. The vast majority of interviewers “protect” themselves against this potential situation by stating the length of the interview during their introductory speeches. When the respondent begins to sound irritated, the interviewers inform the respondents how many minutes have passed since the beginning of the conversation and remind the respondents that they agreed to participate for a specified time. However, interviewers stated that this particular strategy should be used with caution.

While introducing the interview we say that the conversation lasts 10 minutes. It’s true but only if it comes to the respondent who answers efficiently. When somebody thinks too much, the interview extends to 20 minutes. As a result, some respondents hang up. (…) Once it happened to me that the respondent was timing me. I said that the interview lasts 10 minutes. After 10 minutes he said: “I allowed as much time as you wanted; thank you very much, goodbye.” (Woman, 56 years old, CATI interviewer for 1.5 years)

The interviewers claim that respondents’ irritation could lead to the termination of the call. When they sense increasing irritation in respondents’ voices, they try to reassure the respondent that the interview is going to finish soon. Sometimes they also speed up and – when a respondent’s refusal is inevitable – they suggest spreading the interview over several calls and finishing the conversation later.

In the event that the respondent terminates the conversation and hangs up, the interviewers either follow the procedure and make appropriate notes in the system, or they call the respondent one more time, pretending that they experienced a technical failure rather than experiencing the respondent deliberately hanging up. It is surprising that this kind of strategy is often used and – according to the interviewers – proves to be effective.

When the respondent hangs up, I call him again, say that something disrupted telephone service, and repeat the introductory speech. Then he either agrees, or not, but as a rule he feels stupid and says “yes.” (Woman, 25 years old, CATI interviewer for 1.5 years)

When someone hangs up, I call again and ask: “Did you hang up or something disrupted the conversation?” It usually works, but if the respondent hangs up again, I give up. It’s no use scheduling him and making other interviewers’ work difficult. (Woman, 23 years old, CATI interviewer for 2 years)

Some interviewers claim that respondents’ irritation could be also the result of a lack of assertiveness. Such respondents cannot directly articulate that they do not want to participate in the survey. They try to provide excuses and deceive the interviewer, but in the end – under pressure from the interviewer – reluctantly agree to take part in the interview. Afterwards, they are irritated and sometimes hang up.

People often lie. 9 pm, freezing cold outside, an old woman says that she’s leaving. She isn’t leaving, she lacks courage to say “no.” Being assertive is an important thing. (Woman, 50 years old, CATI interviewer for 4 years)

People who don’t want to take part in the interview often say that they’re going out. Even when you’re calling on a cellphone, they say they’re going out. (Man, 52 years old, CATI interviewer for 3 years)

Respondents Having Difficulty in Understanding the Questions

The interviewers unanimously agree that respondents generally find it difficult to understand the questions. In this instance, interviewers sometimes adhere to the following procedure: repeat the questions at a slower pace, mark the “hard to say” option, and in extreme cases thank the respondent and finish the interview.

Sometimes there’re very uncommunicative persons. It’s usually because of their age, but not always. They simply don’t understand what you’re talking to them. That conversation doesn’t make any sense – I’d need to ask all the questions at my own way. If I realize after first screens that someone doesn’t have the faintest about what I’m saying, I give up, finish the interview. (Woman, 61 years old, CATI interviewer for 2.5 years)

However, a majority of interviewers claim that they deviate from interviewing rules when respondents do not understand a particular question. Usually interviewers reformulate the questions, and ask the questions using language that is more easily understood by the respondents. In addition, interviewers reported that they also explain any terms which may be unclear. This type of strategy used by the interviewers seems to be associated with the company at which they work. Although all interviewers are aware of the importance of standardizing the interview protocol, there seems to be unspoken consent to deviate from the rules across most research firms when the respondents appear to experience cognitive difficulties. This consent is usually silent; however, in isolated cases there are supervisors who instruct the interviewers on how to reformulate the questions. Only respondents from two survey companies articulated that their adherence to the standardized interview protocol is strictly monitored.

We often have to simplify the questions. If people don’t understand the questions, they are irritated and want to hang up. Of course, it’s a silent consent. No one is instructed to behave in such way, but no one finds fault with it as well. When I’m talking to a dimwit who doesn’t understand most words, I have two possibilities – either finish the interview or adapt to the respondent. (Man, 24 years old, CATI interviewer for 1 year)

It is worth stating that almost all interviewers indicated that the questions used in the CATI scripts are often formulated using complicated vocabulary and syntax. The questions have not been adapted to suit the intellectual skills of an average respondent. In the interviewers’ opinions, if the researchers placed greater significance on the design of research tools by listening to the interviews and talking to CATI interviewers, there should be no reason for destandardizing the procedure.

Sometimes questions are so horrible that even I don’t know what the matter is. So how can those poor respondents with elementary education understand such questions? (…) No researcher, no supervisor who didn’t work as a CATI interviewer knows things that we – the interviewers – know. (Man, 62 years old, CATI interviewer for 3 years)

Sometimes questions are so long that when I finish reading them, I don’t remember what the beginning is. The respondent – without written text – all the more. (Woman, 69 years old, CATI interviewer for 2 years)

Talkative Respondents

The final type of problem respondent includes talkative respondents who digress and make irrelevant comments about the interview. As the interviewers report, talking to this type of respondent is very troublesome. On the one hand, interviewers feel they must obey the rule of politeness, while on the other, supervisors usually pay attention to the time that is dedicated to a single respondent – an interview that is too long becomes costly both in terms of money and time. As a consequence, the interviewers try to meet both requirements by balancing between being polite and being effective (see Jablonski, 2012; de Ruyter et al., 2001). In extreme cases – similar to instances where respondents demonstrate cognitive difficulties – the interviews are prematurely terminated by the interviewers.

Digressions are the nightmare of all interviewers. The supervisor says that the interview should last 5 minutes. In fact, it lasts 10 minutes – only because of the fact that the respondent is talking at length. I’ve found a way to deal with such persons. I wait until the respondent breathes in, and then I ask the next question. Sometimes it works, sometimes not – some people are so unchangeable that a five-minute interview lasts half an hour. (Man, 24 years old, CATI interviewer for 1 year)

When the respondent is talking at length, I’m filing my nails, reading a book or I press “mute” button and I’m chatting with my friend. I usually let people talk enough, but if it takes so long, I ask the respondent to return to the questions. (Woman, 25 years old, CATI interviewer for 1.5 years)

Interrupting the respondents is risky, it can lead to refusal. We must hear the respondents out, let them talk enough. Sometimes the topic of the research provokes the respondent to swear. In that case I interrupt. I don’t want anyone to show abuse on me. (Woman, 61 years old, CATI interviewer for 2.5 years)

Sometimes I need to hang up. Someone says: “Listen to me, I want to voice my opinion; I’m not going to answer these stupid questions.” When someone is not going to answer stupid questions, then I understand that he’s not interested in the interview. I finish such conversation. (Woman, 25 years old, CATI interviewer for 3.5 years)

Many interviewers also employ another strategy: they suggest that the talkative respondent retains their opinion until the end of the interview when it can be written down.

Sometimes it happens that the respondents at the beginning of the interview say that they have little time and then they start talking at length after each question. In such case I ask them just to answer the question and retain their opinion until the end of the interview. When I finish, nobody even remembers about it. (Woman, 61 years old, CATI interviewer for 2.5 years)

Landline versus Mobile Numbers

The vast majority of interviewers claim that they prefer to administer surveys on mobile rather than on landline phones. As the interviewers report, it is more convenient to conduct surveys using mobile numbers during the introductory part of the interview as well as while asking questions and recording answers. A comprehensive analysis of interviewers’ remarks enables us to list the following factors that can explain this situation.

Mobile phone users are usually younger than landline phone numbers. As it was noted earlier, the quality of the data obtained in telephone surveys can by influenced by respondents’ age.

Contrary to a landline phone, a mobile phone is portable. It is easier to establish contact with the respondent as they usually have their mobiles with them.

Usually a mobile phone is a very personal device – it is the respondent who possesses the phone, not the entire household.

In some mobile surveys the respondent’s name and surname are known in advance – the interviewers know whom they should recruit to the research. It is therefore easier to persuade the respondent to take part in the conversation.

In most mobile surveys it is unnecessary to use within-household respondent selection procedures, which some respondents may find disturbing (see Gaziano, 2005).

However, it must be noted that, according to the interviewers, conducting the interviews via mobile telephony may lead to some problems. The first is the quality of the mobile line, which sometimes is not as good as landline telephony. If the respondent is moving (for instance, driving a car), they are likely to enter an area with poor or no coverage. The interviewers must then call the respondent several times and administer a survey in stages. The interviewers also note that due to the fact that in mobile research the respondents often multitask or are engaged in an important activity which is stopped because of the call, mobile telephone surveys should be shorter than landline ones.

Norwegian and Icelandic Interviewers – Main Differences

In general, CATI interviewers in Norway and Iceland report encountering difficulties while administrating the survey much less frequently than those in Poland. This difference is especially significant for the number of irritated respondents and respondents having difficulties in understanding the questions. In Norway and Iceland, such cases occur so seldom that the interviewers usually fail to elaborate on them and describe strategies they implement in order to deal with problem respondents. We can point out several factors that can explain such differences in interviewers’ experience.

In Icelandic and Norwegian companies that we visited, telephone surveys are conducted using samples of individuals, instead of Random Digit Dialing (which is typical of Poland). As a result, respondents’ details are known in advance. The interviewers have access to all of the respondent’s phone numbers, that are registered, mobile as well as landline, and may choose which one to dial.

Both companies are recognizable and people have confidence in them, especially in Statistics Norway, which is the federal statistical office. The respondents could be more likely to participate in the surveys conducted by well-known organizations.

Norwegian and Icelandic societies differ from the Polish one in terms of education. People there achieve relatively high levels of education and therefore may be more willing to take part in survey research. Moreover, they may experience fewer cognitive difficulties while responding to the questions.

Both companies conduct many surveys that use the same research tools in several waves. This could have an influence on the quality of the questionnaires’ content and reduce the probability of encountering problems while administrating the survey.

Pretesting questionnaires for more understandability is often used at official statistical offices in the Nordic countries.

Conclusions

The results highlight popular strategies used by the interviewers while conducting the interview with problem respondents. One of the most interesting and unexpected findings is the interviewers’ behavior when they encounter respondents who experience difficulties understanding the questions asked. The interviewers deviate from the prescribed protocol to ensure respondents complete the survey. Although such deviation might occur less frequently during telephone interviews than during face-to-face interviews, can have an impact on the reliability of the data obtained. However, Haan et al. (2013) found that many deviations made by interviewers have specific interactional functions; they increase the cohesion and the coherence within the questionnaire and can even enhance response quality.

It is worth mentioning that in the vast majority of survey firms the supervisors allow interviewers to destandardize the interviewing procedure. It is possible that the supervisors share the interviewers’ view and think that the language and syntax used in the questions do not correspond with the intellectual abilities of an average respondent. As we see it, interviewers’ opinions are valuable sources of information about the interview process, and these perspectives should be taken into consideration while designing the CATI script and questionnaires (see Loosveldt, 1997). Moreover, survey researchers should consider creating a standardized set of rules regulating interviewers’ behavior in the event that an interviewer faces the all too common situation of problem respondents.

Additionally, interviewers’ preference for CATI surveys using mobile numbers must be noted. Although some shortcomings of such research are reported, in most cases it is easier for the interviewers to conduct the survey on mobile phones. As noted in the theoretical part of the paper, it is probably not the mode itself, but the factors associated with it that explain this difference.

Footnotes

Funding

Norwegian and Icelandic modules of the project were co-financed by technical assistance funds of the EEA Financial Mechanism and the Norwegian Financial Mechanism within the framework of the Scholarship and Training Fund. Polish module of the project received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

Note

References

AAPOR (American Association for Public Opinion Research) (2010) Cell Phone Task Force Report. New Considerations for Survey Researchers when Planning and Conducting RDD Telephone Surveys in the US with Respondents Reached via Cell Phone Numbers, http://www.aapor.org/AM/Template.cfm?Section=Cell_Phone_Task_Force_Report&Template=/CM/ContentDisplay.cfm&ContentID=3189.

Blumberg

Luke

(2013) Wireless Substitution - Early Release of Estimates from the National Health Interview Survey, July-December 2012. Division of Health Interview Statistics, National Center for Health Statistics, http://www.cdc.gov/nchs/data/nhis/earlyrelease/wireless201306.pdf.

Blyth

(2008) Mixed Mode: The Only ‘Fitness’ Regime? International Journal of Market Research 50(2): 241–66.

Brick

Dipko

Presser

Tucker

Yuan

(2006) Nonresponse Bias in a Dual Frame Sample of Cell and Landline Numbers. Public Opinion Quarterly 70(5): 780–93.

Brick

Dipko

Presser

Tucker

Yuan

(2007) Cell Phone Survey Feasibility in the US - Sampling and Calling Cell Numbers versus Landline Numbers. Public Opinion Quarterly 71(1): 23–39.

Brick

Lepkowski

(2008) Multiple Mode and Frame Telephone Surveys. In Lepkowski

Tucker

Brick

de Leeuw

Japec

Lavrakas

Link

Sangster

(eds) Advances in Telephone Survey Methodology . Hoboken: John Wiley & Sons, 149–69.

Conrad

Schober

(2000) Clarifying Question Meaning in a Household Telephone Survey. Public Opinion Quarterly 64(1): 1–28.

Couper

Groves

(2002) Introductory Interactions in Telephone Surveys and Nonresponse. In Maynard

Houtkoop-Steenstra

Schaeffer

van der Zouwen

(eds) Standardization and Tacit Knowledge. Interaction and Practice in the Survey Interview. New York: John Wiley & Sons, 161–77.

De Leeuw

(1992) Data Quality in Mail, Telephone and Face to Face Surveys. Amsterdam: TT Publikaties, http://edithl.home.xs4all.nl/pubs/disseddl.pdf.

10.

De Leeuw

(2005) To Mix or Not to Mix Data Collection Modes in Surveys. Journal of Official Statistics 21(2): 233–55.

11.

De Leeuw

van der Zouwen

(1988) Data Quality in Telephone and Face to Face Surveys - A Comparative Meta-Analysis. In Groves

Biemer

Lyberg

Massey

Nicholls

II WL

Waksberg

(eds) Telephone Survey Methodology. New York: John Wiley & Sons, p. 283–99.

12.

De Ruyter

Wetzels

Feinberg

(2001) Role Stress in Call Centers: Its Effects on Employee Performance and Satisfaction. Journal of Interactive Marketing 15(2): 23–35.

13.

Dillman

(1978) Mail and Telephone Surveys. The Total Design Method. New York: John Wiley & Sons.

14.

European Commission (2012) E-Communications Household Survey: Special Eurobarometer 381, http://ec.europa.eu/public_opinion/archives/ebs/ebs_381_en.pdf.

15.

Fazzi

Martire

Pitrone

(2009) Hanging by a Thread - The Telephone Interviewers Tell (Their) Story. Proceedings of the Survey Research Methods Section, ASA, 2009: 5603–16.

16.

Frey

(1989) Survey Research by Telephone. Newbury Park: Sage.

17.

Gaziano

(2005) Comparative Analysis of within-Household Respondent Selection Techniques. Public Opinion Quarterly 69(1): 124–57.

18.

Groves

(1979) Actors and Questions in Telephone and Personal Interview Surveys. Public Opinion Quarterly 43(2): 190–205.

19.

Groves

Magilavy

(1986) Measuring and Explaining Interviewer Effects in Centralized Telephone Surveys. Public Opinion Quarterly 50(2): 251–66.

20.

Haan

Ongena

Huiskes

(2013) Interviewers’ Question - Rewording Not Always a Bad Thing. In Winker

Menold

Porst

(eds) Interviewers’ Deviations in Surveys – Impact, Reasons, Detection and Prevention. Frankfurt am Main: Peter Lang, 173–93.

21.

Haeder

(2012) Data Quality in Telephone Surveys via Mobile and Landline Phones. In Haeder

Haeder

Kuehne

(eds) Telephone Surveys in Europe. Research and Practice. Berlin: Springer, 247–62.

22.

Holbrook

Green

Krosnick

(2003) Telephone versus Face-to-Face Interviewing of National Probability Samples with Long Questionnaires. Comparisons of Respondent Satisficing and Social Desirability Response Bias. Public Opinion Quarterly 67(1): 79–125.

23.

Jablonski

(2012) Stressful Situations in Telephone Interviews. Survey Practice 5(4): 1–6, http://www.surveypractice.org/index.php/SurveyPractice/article/view/32/html.

24.

Jordan

Marcus

Reeder

(1980) Response Styles in Telephone and Household Interviewing - A Field Experiment. Public Opinion Quarterly 44(2): 210–22.

25.

Keeter

Kennedy

Clark

Tompson

Mokrzycki

(2007) What’s Missing from National Landline RDD Surveys? The Impact of the Growing Cell-only Population. Public Opinion Quarterly 71(5): 772–92.

26.

Kormendi

(1988) The Quality of Income Information in Telephone and Face to Face Surveys. In Groves

Biemer

Lyberg

Massey

Nicholls

II WL

Waksberg

(eds) Telephone Survey Methodology. New York: John Wiley & Sons, 341–56.

27.

Krosnick

(1999) Survey Research. Annual Review of Psychology 50: 537–67.

28.

Kuehne

Haeder

(2012) Telephone Surveys via Landline and Mobile Phones - Mode Effects and Response Quality. In Haeder

Haeder

Kuehne

(eds) Telephone Surveys in Europe. Research and Practice. Berlin: Springer, 229–46.

29.

Lee

Valliant

(2008) Weighting Telephone Samples Using Propensity Scores. In Lepkowski

Tucker

Brick

de Leeuw

Japec

Lavrakas

Link

Sangster

(eds) Advances in Telephone Survey Methodology. Hoboken: John Wiley & Sons, 170–83.

30.

Loosveldt

(1997) Interaction Characteristics of the Difficult-to-Interview Respondent. International Journal of Public Opinion Research 9(4): 386–94.

31.

Lynn

Kaminska

(2012) Factors Affecting Measurement Error in Mobile Phone Interviews. In Haeder

Haeder

Kuehne

(eds) Telephone Surveys in Europe. Research and Practice. Berlin: Springer, 211–28.

32.

Malhotra

Krosnick

Thomas

(2009) Optimal Design of Branching Questions to Measure Bipolar Constructs. Public Opinion Quarterly 73(2): 304–24.

33.

Mazurkiewicz

(2010) Poland’s Opinion and Market Research Industry. ASK Research & Methods 19: 3–19.

34.

Mohorko

de Leeuw

Hox

(2013) Coverage Bias in European Telephone Surveys - Developments of Landline and Mobile Phone Coverage across Countries and over Time. Survey Methods - Insights from the Field, http://surveyinsights.org/?p=828.

35.

Schober

Conrad

(1997) Does Conversational Interviewing Reduce Survey Measurement Error? Public Opinion Quarterly 61(4): 576–602.

36.

Sykes

Collins

(1988) Effects of Mode of Interview - Experiments in the UK. In Groves

Biemer

Lyberg

Massey

Nicholls

II WL

Waksberg

(eds) Telephone Survey Methodology. New York: John Wiley & Sons, 301–20.

37.

Tarnai

Paxson

(2005) Interviewer Judgments about the Quality of Telephone Interviews. Proceedings of the Survey Research Methods Section, ASA, 2005: 3988–94.

38.

Tucker

(1983) Interviewer Effects in Telephone Surveys. Public Opinion Quarterly 47(1): 84–95.