Do Temporary Dropouts Improve the Composition of Panel Data? An Analysis of “Gap Interviews” in the German Family Panel pairfam

Abstract

In the context of cross-sectional surveys, the scope of research on the impact of response enhancing strategies on sample composition and nonresponse bias is vast. This topic has rarely been addressed for panel studies, however, although these are becoming an increasingly important data source in social research. In this article, we evaluate the impact of reissuing wave nonrespondents on sample composition and survey estimates in the German Family Panel pairfam. In light of concerns about an adequate representation of life changes in panel studies, we focus on whether temporary dropouts improve sample composition in this respect: Using retrospective information from these cases provided at reentry, we approximate the impact of “lost” reports of life changes due to attrition. Our analysis reveals that the inclusion of temporary dropouts does increase sample variability regarding life changes. However, example analyses indicate that substantive conclusions would not be compromised if temporary dropouts were excluded.

Keywords

survey research nonmonotonic panel design attrition life events event history calendar

Maintaining high response rates and survey representativeness as well as avoiding possible errors due to nonparticipation of sample units are issues of central interest to survey researchers. The information necessary to address these issues, however, is scarce. For many years, information on respondents who initially refused to participate in a survey but were then converted to participants, on hard to contact, and on “late” respondents has therefore been used to gain knowledge on nonresponse error (e.g., Filion 1976; Fitzgerald and Fuller 1982; Fulton 2018; Lynn and Clarke 2002; Roberts, Vandenplas, and Stähli 2014; Smith 1984). This strand of research mainly focuses on the impact of extended recruitment efforts on sample composition and nonresponse bias within one wave of cross-sectional surveys.¹ In panel surveys, however, the field period for reissuing nonparticipants is usually extended so as to provide respondents with the possibility to skip one or even several waves before returning to the panel.

In the present study, we evaluate the impact of such “temporary dropouts,” that is, panel respondents who skip one given wave, on sample composition and survey estimates using data from the German Family Panel pairfam (panel analysis of intimate relationships and family dynamics). As capturing life changes is one of the main goals of longitudinal studies, our primary focus is on whether information from temporary dropouts improves sample composition in this respect. The pairfam study is particularly suited for this aim, as it allows for a retrospective assessment of temporary dropout cases based on information they provide in an event history calendar (EHC) after reentering the panel. Thus, we can evaluate whether temporary dropouts report a higher number of life changes during the time in which they did not participate in the panel as compared to respondents without a gap in their panel time line. We focus specifically on reports of changes in relationship status and of childbirth, as avoiding an underreporting of these events is crucial to evaluating the quality of longitudinal studies concerning families and intimate relationships such as pairfam. In addition, we consider the issue from a data user’s perspective and assess the impact of allowing for temporary dropouts on substantive regression estimates. Alongside these analyses, we evaluate the contribution of temporary dropouts to an increase or decrease in selectivity regarding baseline sample characteristics. Finally, we provide summary information on effects of reissuing temporary dropouts for overall sample representativeness by using recently developed indicators for representative response.

Patterns of Panel Survey Response and Sample Composition

Within the survey field period of both cross-sectional and panel studies, various measures can be implemented to convert initial nonrespondents, for example, increasing the number of contact attempts or using incentives. Following the aim to keep response rates high, many panel surveys additionally allow for a nonmonotonic design, meaning respondents who did not participate in a given wave are refielded in the next wave. Due to data protection laws in Germany, usually only noncontacts and soft refusals may be reissued. Such nonparticipants who then return in a later wave are termed temporary dropouts, as opposed to respondents who permanently leave the panel (permanent dropouts or “attriters”).

Compared to monotonic designs in which onetime nonparticipants are discarded from the address pool, nonmonotonic designs are more costly: Tracking nonrespondents tends to incur more expenses, the questionnaire may become more complex as it could imply a special routing of nonmonotonic cases, and other tasks such as calculating weights require increased effort.² The main advantage of a nonmonotonic design, however, is that more respondents remain in the panel, that is, the panel dies out at a slower rate. Further, in addition to higher response rates, reissuing temporary dropouts could possibly have a beneficial impact on sample composition by including a distinct respondent group, which in turn increases sample variability.

Following general approaches to assessing nonresponse bias (for an overview, see Groves 2006), the impact of respondents who participated only after increased efforts on nonresponse bias and sample composition can be evaluated by comparing estimates based on samples including and excluding these cases to an external benchmark (e.g., from administrative data or a “gold standard” survey). As such information is often missing for key survey variables, more indirect approaches for assessing nonresponse bias are applied, for which respondent groups within a given survey are differentiated (Groves 2006:654-56).³ As one example, the “continuum of resistance” argument has been applied in the context of cross-sectional surveys to evaluate possible beneficial effects of including converted nonrespondents on sample composition (see Filion 1976; Lin and Schaeffer 1995; Stoop 2005). Respondents are placed on a continuum representing the amount of effort required to interview them: easy to be interviewed (e.g., those who participate after the first contact) at the one end and nonrespondents at the other. Respondents who require increased field efforts are then assumed to be more similar to nonparticipants, as without these efforts they would not have been interviewed (Lin and Schaeffer 1995).

When applying the continuum of resistance argument to nonmonotonic panel designs, the framework suggests that temporary dropouts more closely resemble permanent dropouts rather than continuous respondents and that consequently a nonmonotonic design is able to retain participants who resemble nonrespondents in the sample for a longer period of time. Recent studies support this notion, showing that temporary dropout cases indeed differ from continuous respondents (Michaud et al. 2011; Voorpostel 2010; Watson and Wooden 2014). Using data from three national household panel studies, Watson and Wooden (2014) found that factors influencing the decision to reengage differ considerably from those affecting continued participation. In an analysis of attrition patterns in the Swiss Household Panel, Voorpostel (2010) showed that respondents who reentered the study more closely resembled respondents who dropped out permanently in several characteristics than they did continuous respondents. Similarly, Michaud et al. (2011) found that temporary nonrespondents in the Health and Retirement Study differ in various characteristics from the “always in” respondents and that their inclusion helps reduce nonresponse bias.

These results indicate that nonmonotonic designs might indeed increase sample variability to a certain extent. However, the studies mentioned are based on information from wave 1 or the waves prior to temporary dropout only. In this article, we aim to extend this research by focusing on reports of life changes across the panel. Previous studies have found that specific life changes tend to increase nonresponse in longitudinal surveys (Fitzgerald, Gottschalk, and Moffitt 1998; Lemay 2009; Müller and Castiglioni 2015; Trappmann, Gramlich, and Mosthaf 2015; Voorpostel and Lipps 2011) and that continuous respondents tend to experience fewer life changes than respondents with nonmonotonic response patterns (Voorpostel and Lipps 2011). These findings raise concerns of an underrepresentation of life changes in panel studies. Based on this research, and while applying the continuum of resistance argument, we hypothesize that temporary dropouts refused participation during times in which they experienced certain life changes (e.g., entering a period of cohabitation and/or marriage, partnership dissolution, and birth of a child), and in that respect more closely resemble attriters than continuous respondents. We investigate whether temporary dropouts exhibit more life-course transitions around the time of nonparticipation as compared to continuous respondents in the corresponding period. For this purpose, we use information provided by temporary dropouts after they reentered the panel (from a “gap interview”). If our hypothesis holds true, refielding temporary dropouts will prove to be a worthwhile measure to increase the number of life-course transitions observed in panel data and, thus, to improve sample composition.

Data and Analytical Approach

pairfam Data

Our analysis relies on data from the first seven waves of the German Family Panel pairfam (Brüderl et al. 2016) and paradata including reasons for nonparticipation from waves 2 to 6. The pairfam study is designed as an annual survey of a random sample of German residents from three different birth cohorts: 1971–1973, 1981–1983, and 1991–1993. Data collection started in 2008 with approximately 4,000 interviews from each cohort by computer-assisted personal interviews. The focus of the study is on partnership dynamics and dissolution, fertility attitudes and generative behavior, parenting and child development, and intergenerational relationships. A more detailed description of the study is provided by Huinink et al. (2011).

The pairfam questionnaire includes a time line based calendar instrument. In monthly intervals, this EHC captures information on respondents’ relationships, children, places of residence, and education and employment history since the last interview. The EHC graphically collects and displays transitions in these life domains over the past year and can be edited directly by respondents. As a starting point, preloaded information from the last interview is presented (for more details, see Brüderl et al. 2017).

The pairfam design allows for at most one “gap year” between two interviews, that is, after two consecutive waves of nonparticipation, respondents are discarded from the address pool. Beginning with wave 3, all respondents who could not be contacted in the previous wave, as well as soft refusals, are refielded in the next year. These temporary dropouts are surveyed with an extended version of the EHC, retrospectively covering the complete time span since the last interview (approximately two years instead of one). This feature provides an implicit gap interview and to our knowledge is unique to the pairfam panel study (Brüderl et al. 2017).

Analytical Approach

Our analyses address the following issues: First, whether and how temporary dropouts in the pairfam study differ from continuous respondents. Second, and provided the two groups differ, we investigate possible implications of allowing for temporary dropout on sample composition and substantive results.

To evaluate the contribution of temporary dropout data to increasing or decreasing sample selectivity with respect to baseline characteristics, we first classify respondents into one of the three categories based on their response status up to wave 6. The first contains respondents who continuously participate in waves 1–6, whereas the second includes those who skipped at least one wave between waves 1 and 6, and the third comprises all permanent dropouts, that is, respondents who continuously participate in any number of waves and then leave the panel permanently prior to or in wave 6. We use wave 7 information to determine whether a dropout in wave 6 is permanent or temporary. Respondents with a gap in their panel time line, who then drop out permanently, are classified as temporary dropouts. Given these three groups, we apply a multinomial logistic regression to investigate which factors affect response status using wave 1 characteristics (for similar approaches, see Michaud et al. 2011; Voorpostel and Lipps 2011).

Next, we extend our focus to reports of life changes using the retrospective information provided with the EHC tool. We evaluate whether temporary dropouts exhibit more transitions of partnership formation, partnership dissolution, moving in together, marriage, or childbirth between waves t − 2 and t − 1 than do continuous respondents in the same time span. For respondents with a gap, wave t − 2 denotes the wave prior to dropout, wave t − 1 the gap wave, and wave t the reentry wave (Figure 1). In order to use comparable time spans for both groups, a fictive interview date for each temporary dropout must be set for the gap wave; according to our definition, this interview date is 12 months prior to the date of their reentry to the panel. Thus, for temporary dropouts, we artificially set the time period used for comparison with continuous respondents to begin with the interview date in the wave prior to dropout (wave t − 2) and end at the date of reentry (wave t) minus one year.

This approach implies that for all temporary dropout cases, we use information provided at reentry which covers a time period that, on average, refers to facts dating back 12 months further than the time period for which continuous respondents provide information. Thus, we cannot exclude the possibility of increased recall problems among temporary dropouts, which could lead to an underreporting of changes among this group. As pairfam’s EHC implements dependent interviewing with preloads from the previous interview and deals with factual information, we presume this to be a minor issue. However, in the case of recall problems, our approach would provide a conservative estimate of differences between the two groups, as we expect respondents completing a longer EHC to be more likely to underreport changes.

In order to evaluate the substantive impact of allowing for temporary dropouts, we compare different strategies for dealing with information from such cases in example analyses. More specifically, we apply discrete-time event history models for specific events to three different sample definitions, respectively, varying the amount of temporary dropout information taken into account. In the first model, we use all information provided by temporary dropouts, including the gap interviews. Values for the missing person-year are imputed as follows: Time constant variables stem from the wave prior to temporary dropout, while time-varying characteristics are drawn from retrospective EHC information as provided in the reentry wave.⁴ In the second model, temporary dropouts reenter the risk set for our analyses when reentering the panel, but values for the missing person-year are not imputed. This data structure corresponds to a nonmonotonic design without a gap interview. In a third model, we treat temporary dropouts as attriters, excluding the wave they return to the panel and all following waves, thereby simulating a monotonic design. The impact of allowing for temporary dropouts on substantive regression estimates can then be evaluated by comparing the results from the models estimated with these three samples (for a similar approach, see Michaud et al. 2011). We specifically focus on characteristics that have been identified to affect response status in our first analysis.

Finally, we evaluate the impact of reissuing temporary dropouts on sample composition by way of a representativeness (R-)indicator (Schouten, Cobben, and Bethlehem 2009). The R-indicator is based on estimated response propensities ${\hat{p}}_{i}$ , which are thought to vary across groups of respondents. Variation is measured by the standard deviation ${\hat{S}}_{p}$ of response propensities.⁵ The R-indicator is defined as ${\hat{R}}_{p} = 1 - 2 {\hat{S}}_{p}$ and assumes values between 0 and 1, whereby 0 corresponds to maximal variation in response propensities and the lowest level of representativeness and 1 to a situation in which response propensities show no variation, indicating strong representativeness (Schouten et al. 2009). It is important to note that representativeness based on this R-indicator can only be evaluated relative to the variables used to estimate the response propensities. Representativeness thus refers to a lack of association between response and these specific variables (Schouten et al. 2009:105-6). When evaluating representativeness of a given survey over time, these auxiliary variables including paradata should ideally relate to key survey variables and survey topic (Schouten et al. 2012:388).

Rather than using a reference sample deemed representative (e.g., the target population in wave 1), we use the R-indicator in a relative sense, comparing different panel study designs and their respective samples. For each panel wave from 3 to 6, we apply the R-indicator to two survey samples. “Sample 1,” our benchmark, results from a nonmonotonic design, that is, from reissuing temporary dropouts from wave t − 1 at wave t (as is done in pairfam). “Sample 2” simulates a situation in which these cases have not been reissued, corresponding to a monotonic design. Here, temporary dropout information from waves prior to t − 1 is also not considered.

Computation of the R-indicator proceeds in two steps. First, we run logistic regression models to estimate response propensities in samples 1 and 2 for each wave, meaning the propensity of both respondents and temporary dropouts from wave t − 1 to respond in wave t (sample 1) and the propensity of respondents in wave t − 1 to respond in wave t (sample 2). Estimation of response propensities is based on variables that relate to response and are of substantive interest including age, relationship status, at least one child aged 0–14, years of education, migration background, home ownership, and employment status. Furthermore, we include paradata on interviewers’ estimation of respondents’ willingness to participate in the next wave as well as an indicator for a match of interviewer and respondent sex. Variables stem from wave t − 1 for respondents in waves t − 1 and t − 2 for temporary dropouts. In a second step, the R-indicators for samples 1 and 2 are calculated based on the standard deviation of the respective estimated response propensities.⁶

Results

Descriptive Measures of Temporary Dropouts in pairfam

As the pairfam design allows for only one gap year, the number of temporary dropouts is small: 518 of wave 2 nonparticipants returned in wave 3, representing 6.6 percent of all wave 3 respondents (n = 7,901). The proportion of temporary dropouts who return in each year’s collected sample then declines over the panel, representing 4.6 percent (n = 321) of wave 4, 4.5 percent (n = 279) of wave 5, 4.1 percent (n = 236) of wave 6, and 3.3 percent (n = 169) of all wave 7 respondents (see Online Appendix Table A1). Correspondingly, response rates of temporary dropouts at reentry are low: The response rate in wave 3 of wave 2 temporary dropouts, for instance, comes to 17.4 percent, as compared to 81.4 percent of wave 2 participants.⁷ Temporary dropouts also exhibit higher attrition rates after their return to the panel: For example, the wave 4 attrition rate of temporary dropouts from wave 2 amounts to 23.9 percent, compared to 11.5 percent in the overall sample.⁸ After their return, however, the majority of temporary dropouts from waves 2-6 remains in the panel until wave 7 (7.0 percent of the initial wave 1 sample). In contrast, only 4.5 percent of temporary dropouts from waves 2-6 of the wave 1 sample drop out permanently in the waves after their return and prior to wave 7. In summary, the small proportion of temporary dropouts in combination with a higher attrition rate after their return to the panel leads to a relatively small informative contribution from these cases to the pairfam data: 11.6 percent of the overall number of person-year observations in waves 3–7 originate from temporary dropouts after their reentry.

When considering reasons for nonparticipation, that is, noncontact and soft refusal, temporary dropout is more closely associated to a failure to trace or to contact respondents, whereas the primary reason for permanent attrition is refusal. Averaged over waves 2–6, 56.7 percent of temporary dropout cases are due to noncontact, which occurs far more rarely among permanent dropouts (24.6 percent, see Online Appendix Table A1).

Response Patterns

Figure 2 summarizes the results from a multinomial logistic regression estimation of response status after wave 6. In this model, response status is explained by respondents’ characteristics from wave 1 (Online Appendix Table A2). To simplify the interpretation, we present average marginal effects (AMEs) of the probability to belong to the temporary or permanent dropout group as compared to continuous participation as the baseline outcome, respectively (the complete results can be found in Online Appendix Table A3).⁹ Note that the sum of the AMEs for each independent variable across all three groups is always zero.

Overall, we do not observe pronounced selectivity patterns. Permanent dropouts do not appear to significantly differ from continuous respondents for most of the observed characteristics. Only years of education and migration status show strong effects on response status. Consequently, the contribution of temporary dropouts to an increase or decrease in selectivity can be assessed: The education effect for permanent dropouts is strongly negative, indicating that respondents with less years spent in education are more likely to leave the panel. In contrast, no selectivity along education can be observed for temporary dropouts, which leads us to conclude that they constitute a more balanced sample in this respect. We can hence state that allowing for a nonmonotonic design increases sample variability, as relatively many less educated respondents reenter the panel who would otherwise be excluded from a monotonous sample design. The same applies for migration status: Respondents with a migration background exhibit a higher likelihood of permanent dropout compared to those without. Temporary dropouts, however, constitute a balanced sample concerning migration background. Including these cases therefore improves the sample representation of migrants in the panel. Temporary dropouts also appear to balance the proportion of respondents in a living apart together (LAT) relationship, as well as the distribution of employment statuses (Figure 2), although effects are not quite as pronounced as they are for education and migration status.¹⁰

Comparison of the Number of Life-course Transitions

In a next step, we use EHC information to evaluate whether temporary dropouts experience more life-course transitions than do continuous respondents. Figure 3 displays the number of transitions experienced in the gap year (between the wave prior to dropout and the reentry date minus one year) per 100 temporary dropouts and that of continuous respondents for the same time span.

The relative number of transitions does not differ in the case of beginning a new relationship, marriage, or the birth of a child (Figure 3). However, temporary dropouts report a higher number of separations and transitions to cohabitation than do continuous respondents in the corresponding time span, experiencing a separation with 1.9 percentage points and a beginning of a new cohabitation period with 2.5 percentage points more often. Two-sample t-tests indicate that differences in the respective means for separation are significant at the .05 level and for cohabitation at the .001 level. Given that on average, half of the respondents who experience these events change their place of residence, we presume that temporary dropout is also related to relocation. Correspondingly, when differentiating the number of separations by relationship status prior to separation (LAT vs. married and nonmarried cohabitation), temporary dropouts from both subgroups report more separations, but differences are only significant (level .001) for respondents who were previously cohabiting with their partner (results not shown). A complementary assessment of the number of transitions of education, employment status, and nonworking activities reveals higher numbers for temporary dropouts throughout, with differences most pronounced for education (Online Appendix Figure A1). We presume this to be due to educational transitions (e.g., from secondary school to university or from any education status to the labor market) often being highly correlated with residential changes. Evaluating reported changes in main residence uncovers even more pronounced differences between temporary dropouts and continuous respondents, with such transitions more than three times more frequent among the former group (Online Appendix Figure A1).

The overall finding that temporary dropouts report a higher number of life-course transitions related to moving residence corresponds to the fact that tracking respondents after a move requires more time and effort, and consequently these respondents are more often classified as temporary dropouts. This also becomes evident when comparing reasons for nonparticipation (i.e., soft refusal and noncontact) between temporary and permanent dropouts, with nonresponse due to noncontact far more frequent among temporary than permanent dropouts (Online Appendix Table A1).

Impact on Substantive Regression Estimates

We now simulate how the results of an example discrete-time event history analysis on separation and cohabitation transitions would change if we would not allow for temporary dropout cases. Analyses of transitions to partnership dissolution only comprise respondents at risk of experiencing this transition, that is, those in a relationship. Similarly, analysis samples for transitions into cohabitation are restricted to respondents in a LAT relationship. For both transitions, only the first relationship recorded within the observation period is considered.

For transitions to separation (Table 1), the pattern of substantive regression estimates is largely similar across the three different samples including and excluding temporary dropout information. Model 1 analyzes all information available in pairfam, including gap year information. Strong effects are found for relationship duration, children and/or pregnancy, and education. With increasing relationship duration, the rate of partnership dissolution decreases. The same is true for having children or expecting a child. Furthermore, better educated respondents exhibit a lower rate of separation. If we were to subsequently drop information provided by temporary dropouts after their return to the panel (model 2), the estimations remain mostly stable. As model 2 does not include information from the retrospective EHC covering the gap year, some respondents (n = 17), person-years (n = 795), and events (n = 34) are lost. However, the results do not change significantly. Similarly, if we exclude all information provided by temporary dropouts after their reentry, simulating a monotonic design (model 3), we lose another 1,734 person-years and 222 events while estimates remain largely stable. Marginal differences in effects across models can only be observed for years of education and migrations status, which have both been found to affect response status (Figure 1). The education effect is slightly less pronounced when excluding observations from temporary dropouts after reentry (model 3), but significance levels remain stable across all models. The effect of a second-generation migration background is marginally significant only when including temporary dropout information (models 1 and 2).

Table 1.

Logistic Regression Estimations of the Transition to Separation and Various Sample Definitions; Average Marginal Effects.

	(1)	(2)	(3)
	Including Information from Gap Interview	Excluding Information from Gap Interview	Excluding all Information after Reentry
Log. relationship duration (years)	−.041*** (.001)	−.042*** (.001)	−.040*** (.001)
Children and/or expecting child	−.027*** (.004)	−.026*** (.004)	−.026*** (.004)
Attitude: marriage lifelong commitment	−.008** (.003)	−.007** (.003)	−.006* (.003)
Years of education/10	−.023*** (.006)	−.024*** (.006)	−.020*** (.006)
Migration status (ref.: nonmigrant)
First-generation migrant	−.007 (.005)	−.007 (.005)	−.006 (.005)
Second-generation migrant	−.008^† (.004)	−.007^† (.004)	−.006 (.004)
Home ownership	−.008** (.003)	−.008** (.003)	−.008** (.003)
Residence: periphery or urban area (ref. rural)	−.005^† (.003)	−.004 (.003)	−.006^† (.003)
Religion (ref.: no religion)
Catholic	−.009** (.003)	−.011** (.004)	−.009* (.004)
Protestant	−.012*** (.003)	−.012*** (.003)	−.011*** (.003)
Islam	−.016* (.007)	−.018* (.007)	−.016* (.007)
Other religion	−.029*** (.007)	−.029*** (.007)	−.026*** (.007)
Pseudo R²	0.193	0.197	0.200
N (respondents)	9,119	9,102	8,926
N (person-years)	28,595	27,800	26,066
N (events)	1,879	1,845	1,623

Note: All samples are comprised of respondents at risk of separation, that is, who are in a relationship (only the first relationship within the observation period is considered); waves 1–6. Models additionally include respondent age. Standard errors are in parentheses.

^†p < .10. *p < .05. **p < .01. ***p < .001.

Different approaches to estimating effects on entering into a period of cohabitation also yield similar results (Table 2): With increasing relationship duration, the rate of transitioning to cohabitation increases. The same is true if a child is expected. Furthermore, better educated respondents enter into cohabitation periods at a higher rate, whereas being enrolled in education or vocational training decreases the cohabitation rate. Differences in effects across models can be found for relationship duration and expecting a child, with more pronounced effects for models including temporary dropout information.

Table 2.

Logistic Regression Estimations of the Transition to Cohabitation and Various Sample Definitions; Average Marginal Effects.

	(1)	(2)	(3)
	Including Information from Gap Interview	Excluding Information from Gap Interview	Excluding all Information after Reentry
Log. relationship duration (years)	.061*** (.004)	.063*** (.004)	.057*** (.004)
Anchor/partner expecting child	.096*** (.015)	.096*** (.015)	.081*** (.015)
Attitude: marriage if permanently cohabiting	−.001 (.007)	−.000 (.007)	−.003 (.007)
Years of education/10	.032* (.013)	.030* (.014)	.033* (.013)
Enrollment school/vocational qualification	−.026** (.009)	−.027** (.009)	−.027** (.009)
Migration status (ref.: non migrant)
First-generation migrant	−.002 (.012)	−.004 (.012)	−.011 (.012)
Second-generation migrant	.002 (.012)	.004 (.011)	−.002 (.011)
Born in East Germany	−.016 (.012)	−.018 (.012)	−.021^† (.012)
Residence: rural area (ref.)
Periphery	.014 (.009)	.019* (.009)	.017^† (.009)
Urban area	.008 (.008)	.012 (.009)	.012 (.008)
Religion (ref.: no religion)
Catholic	−.007 (.010)	−.008 (.010)	−.009 (.010)
Protestant	.007 (.009)	.005 (.010)	.004 (.010)
Islam	.006 (.019)	.003 (.019)	.008 (.021)
Other religion	−.021 (.020)	−.021 (.021)	−.011 (.022)
Pseudo R²	0.172	0.183	0.173
N (respondents)	4,051	4,032	3,893
N (person-years)	7,502	7,304	6,944
N (events)	766	743	651

Note: All samples are comprised of respondents at risk of entering into cohabitation, that is, those in a living apart together relationship (only the first relationship within the observation period is considered); waves 1–6. Models additionally include respondent age. Standard errors are in parentheses.

^†p < .10. *p < .05. **p < .01. ***p < .001.

To summarize, although including temporary dropouts appears to be worthwhile in terms of increasing the number of observations and life-course transitions available for substantive analysis purposes, excluding these cases would not—at least for these specific pairfam examples—substantially alter results of a multivariate estimation. However, we do find marginal differences across models including and excluding temporary dropout information, in particular regarding characteristics found to affect response status.

Indicators of Representativeness

As presented thus far, our analyses show that pairfam’s temporary dropouts differ from continuous respondents along several baseline characteristics and experience more transitions in certain life domains. However, the total number of these cases seems to be too small to significantly affect substantive analyses. The last step of our analysis aims at summarizing the impact of reissuing temporary dropouts on overall sample composition by means of R-indicators calculated for panel waves 3–6 (Table 3). R-indicators based on our benchmark sample 1 (reissuing temporary dropouts from wave t − 1 at wave t) and sample 2 (simulating a monotonic sample) show differences in the variation of response propensities for age, relationship status, at least one child aged 0–14, years of education, migration background, home ownership, employment status, interviewers’ estimation of respondents’ willingness to participate in the next wave, and match of interviewer and respondent sex. Relative to these variables, sample representativeness hypothetically decreases if wave 2 temporary dropouts would not be reissued in wave 3 (Table 3). However, differences between including and excluding these cases are not significant. For the following waves, differences between the two scenarios are less pronounced (wave 5), or not existent (wave 4), which could be due to a decreasing number of temporary dropouts in each consecutive wave. In wave 6, the R-indicator shows higher representativeness for a monotonic sample. We presume that effects counteracting selectivity (e.g., on migration background) by means of including temporary dropouts are more pronounced in the first waves of a panel study, as selectivity along these variables due to attrition is more pronounced at the beginning of the panel. Furthermore, the effect of reissuing temporary dropouts on sample representativeness for waves 4–6 is confounded by the effect of excluding temporary dropouts from previous waves. When simulating a monotonic design (Table 3, monotonic sample), respondents from the latter group are excluded from the sample. However, if excluding all observations from temporary dropouts from previous waves results in a more homogeneous sample concerning response propensities, R-indicators will increase. Therefore, to disentangle the effects of reissuing temporary dropouts from wave t − 1 at wave t, and of excluding temporary dropout cases from wave t − 2 and earlier, we additionally compare R-indicators calculated from our benchmark sample to those obtained when keeping reentry observations from temporary dropouts in the sample, which for panel waves 4–6, we denote as the “wave-specific monotonic sample” (Table 4). Here, we observe lower levels of representativeness in the case of not reissuing temporary dropouts throughout panel waves 3–5, although differences are not significant.

Table 3.

Effects of Reissuing Temporary Dropouts on Sample Representativeness Compared to Monotonic Sample.

Wave t		R-indicator	95% Confidence Interval		Mean Response Propensity	N
Wave t		R-indicator	Lower	Upper	Mean Response Propensity	N
Wave 3	Reissuing W2 dropouts	.85	.83	.87	.86	9,239
Wave 3	Monotonic sample	.83	.81	.85	.82	8,734
Wave 4	Reissuing W3 dropouts	.86	.84	.88	.89	7,911
Wave 4	Monotonic sample	.86	.83	.88	.85	7,098
Wave 5	Reissuing W4 dropouts	.93	.91	.95	.90	6,959
Wave 5	Monotonic sample	.92	.90	.94	.87	6,045
Wave 6	Reissuing W5 dropouts	.91	.89	.93	.91	6,233
	Monotonic sample	.92	.90	.94	.89	5,245

Note: R-indicators are calculated based on the response propensity in wave t of respondents and temporary dropouts from wave t − 1 (“reissuing t − 1 dropouts”) and response propensity in wave t of respondents in wave t − 1, excluding temporary dropouts from wave t − 2 and earlier (“monotonic sample”). Response propensities are estimated based on age, relationship status, at least one child 0–14, years of education, migration background, home ownership, employment status, interviewers’ estimation of respondents’ willingness to participate in the next wave, and a match of interviewer and respondent sex.

Table 4.

Effects of Reissuing Temporary Dropouts on Sample Representativeness Compared to Wave-specific Monotonic Sample.

Wave t		R-indicator	95% Confidence Interval		Mean Response Propensity	N
Wave t		R-indicator	Lower	Upper	Mean Response Propensity	N
Wave 3	Reissuing W2 dropouts	.85	.83	.87	.86	9,239
Wave 3	Monotonic sample	.83	.81	.85	.82	8,734
Wave 4	Reissuing W3 dropouts	.86	.84	.88	.89	7,911
Wave 4	Wave-specific monotonic sample	.85	.83	.87	.84	7,603
Wave 5	Reissuing W4 dropouts	.93	.91	.95	.90	6,959
Wave 5	Wave-specific monotonic sample	.91	.89	.93	.86	6,692
Wave 6	Reissuing W5 dropouts	.91	.89	.93	.91	6,233
Wave 6	Wave-specific monotonic sample	.91	.89	.93	.88	6,006

Note: R-indicators are calculated based on the response propensity in wave t of respondents and temporary dropouts from wave t − 1 (“reissuing t − 1 dropouts”) and response propensity in wave t of respondents in wave t − 1, including temporary dropouts from wave t − 2 and earlier who participate in wave t − 1 (“wave-specific monotonic sample”). Response propensities are estimated based on age, relationship status, at least one child 0–14, years of education, migration background, home ownership, employment status, interviewers’ estimation of respondents’ willingness to participate in the next wave, and a match of interviewer and respondent sex.

It is important to note that if we were to compare levels of representativeness over time, results would indicate a more positive picture of sample representativeness for later panel waves. While this may appear counterintuitive at a first glance, these results are most likely due to a decreasing variation in response propensities with each additional wave caused by attrition. Therefore, we suggest restricting the evaluation of sample representativeness based on R-indicators for this longitudinal example to comparing samples including and excluding temporary dropouts by wave.

To summarize, temporary dropouts differ from continuous respondents in several respects. Generally, those who skip one wave are less educated, more often have a migration background, and exhibit more transitions when it comes to partnership dissolution and new cohabitation periods. Thus, the inclusion of temporary dropouts increases sample variability in the pairfam data. However, results from a substantive regression estimation are virtually not affected by excluding these cases. Similarly, sample representativeness does not appear to be severely compromised when simulating a monotonic design. This is most likely due to the fact that the information provided by temporary dropouts after their reentry constitutes only a small amount of the overall information available in the panel.

Discussion and Conclusions

In this article, we evaluate whether a nonmonotonic design increases sample variability. Whether allowing for temporary dropout cases has any effect on sample composition and selectivity bias depends on whether these respondents differ from those who participate continuously in each wave. Our findings indicate that this is indeed the case for baseline characteristics such as education level and migration background. Extending analyses to estimates of life changes provides further evidence to this, as temporary dropouts report significantly more changes during the time they were absent from the panel as compared to continuous respondents in the same time span. This leads us to conclude that refielding temporary dropouts is an effective way of improving sample variability in panel data, especially by way of increasing the number of life-course transitions observed. Considering that an underreporting of the number of life changes is a major concern to panel studies, our results suggest that offering respondents the possibility to skip one or several waves might counteract this bias to a certain extent. This is especially relevant for survey populations thought to experience a high number of life-course transitions related to changing residence during the panel: The pairfam sample is comprised of a relatively young respondent group which passes through various life stages (e.g., leaving the parental home, entering the labor market, and starting a relationship or family) and is therefore particularly prone to panel dropout. As temporary dropout seems to be due to transitory circumstances rather than a lack of willingness to continue participation, applying nonmonotonic designs appears to be even more advisable for panel studies addressing such samples and topics. That said, it is important to note that findings based on the pairfam cohort study, targeting a young, highly mobile population may not translate directly to a population-level sample. There are several further issues to be aware of when considering our conclusions.

First, assessing the impact of allowing for temporary dropouts in terms of life changes is challenging, as there is no external benchmark available for changes such as separation and moving in together as captured in the pairfam data. Thus, it is difficult to quantify improvements of sample composition. Evaluating the impact of temporary dropouts on sample composition can therefore only be seen as an approximation and is based on the assumption that temporary dropouts more closely resemble nonparticipants than continuous respondents.

Second, a possible increase in the accuracy of survey estimates does depend not only on the magnitude of differences between temporary dropouts and continuous participants but also on their proportion in the sample. In pairfam, the amount of information provided by temporary dropouts after their reentry is small when compared to the rest of the sample. Thus, a higher number of life changes observed from this group along with a higher variability in characteristics such as education and migration background does not appear to substantially affect substantive regression estimates in our example. In addition, drawing generalizable conclusions based on such a small respondent group is difficult.

However, as we do find marginal differences across models including and excluding temporary dropout information, in particular with regard to characteristics found to affect response status, we recommend that data users clearly document their sample choice should they decide to exclude data from temporary dropouts. In our opinion, including temporary dropouts is generally a wise choice, as these cases tend to offer more intraindividual variation, which is what researchers generally seek to analyze in a longitudinal analysis. Furthermore, retaining temporary dropouts could help reduce selectivity over time constant characteristics such as migration background or education, which remains almost constant over the pairfam sample. Losing information from migrants and lesser educated respondents would consistently reduce the sample’s representativeness regarding these variables with each further wave.

Supplemental Material

Online_Appendix - Do Temporary Dropouts Improve the Composition of Panel Data? An Analysis of “Gap Interviews” in the German Family Panel pairfam

Online_Appendix for Do Temporary Dropouts Improve the Composition of Panel Data? An Analysis of “Gap Interviews” in the German Family Panel pairfam by Bettina Müller and Laura Castiglioni in Sociological Methods & Research

Footnotes

Acknowledgements

This article uses data from the German Family Panel (pairfam) coordinated by Josef Brüderl, Karsten Hank, Johannes Huinink, Bernhard Nauck, Franz Neyer, and Sabine Walper. The pairfam is funded as a long-term project by the German Research Foundation (DFG). We are grateful to Josef Brüderl for helpful contributions to an earlier version of this paper.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Supplemental Material

Supplemental material for this article is available online.

Notes

References

Brüderl

Josef

Castiglioni

Laura

Ludwig

Volker

Pforr

Klaus

Schmiedeberg

Claudia

. 2017. “Collecting Event History Data with a Panel Survey: Combining an Electronic Event History Calendar and Dependent Interviewing.” Methods, Data, Analyses 11:45–66.

Brüderl

Josef

Hank

Karsten

Huinink

Johannes

Nauck

Bernhard

Neyer

Franz J.

Walper

Sabine

Alt

Philipp

Buhr

Petra

Castiglioni

Laura

Fiedrich

Stefan

Finn

Christine

Hajek

Kristin

Herzig

Michel

Huyer-May

Bernadette

Lenke

Rüdiger

Müller

Bettina

Peter

Timo

Salzburger

Veronika

Schmiedeberg

Claudia

Schütze

Philipp

Schumann

Nina

Thönnissen

Carolin

Wetzel

Martin

Wilhelm

Barbara

. 2016. “The German Family Panel (pairfam).” GESIS Data Archive, Cologne, Germany. ZA5678 Data File Version 7.0.0. doi: 10.4232/pairfam.5678.7.0.0.

Burton

Jonathan

Laurie

Heather

Lynn

Peter

. 2006. “The Long-term Effectiveness of Refusal Conversion Procedures on Longitudinal Surveys.” Journal of the Royal Statistical Society Series A 169:459–78.

De Heij

Vincent

Schouten

Barry

Shlomo

Natalie

. 2015. “RISQ manual 2.1. Tools in SAS and R for the Computation of R-indicators, Partial R-indicators and Partial Coefficients of Variation.” Retrieved September 28, 2016 (http://www.risq-project.eu/tools21/RISQ-manual-v21.pdf).

Filion

F. L.

1976. “Exploring and Correcting for Nonresponse Bias Using Follow-ups of Nonrespondents.” The Pacific Sociological Review 19:401–8.

Fitzgerald

Robert

Fuller

Linda

. 1982. “I Hear You Knocking But You Can’t Come In: The Effects of Reluctant Respondents and Refusers on Sample Survey Estimates.” Sociological Methods & Research 11:3–32.

Fitzgerald

John

Gottschalk

Peter

Moffitt

Robert

. 1998. “An Analysis of Sample Attrition in Panel Data: The Michigan Panel Study of Income Dynamics.” The Journal of Human Resources 33:251–99.

Fulton

Brad. R

. 2018. “Organizations and Survey Research: Implementing Response Enhancing Strategies and Conducting Nonresponse Analyses.” Sociological Methods & Research 47:240–76. doi: 10.1177/0049124115626169.

Groves

Robert M.

2006. “Nonresponse Rates and Nonresponse Bias in Household Surveys.” Public Opinion Quarterly 70:646–75.

10.

Huinink

Johannes

Brüderl

Josef

Nauck

Bernhard

Walper

Sabine

Castiglioni

Laura

Feldhaus

Michael

. 2011. “Panel Analysis of Intimate Relationships and Family Dynamics (pairfam): Conceptual Framework and Design.” Zeitschrift für Familienforschung 23:77–101.

11.

Lemay

Michael

. 2009. Understanding the Mechanism of Panel Attrition. PhD dissertation, University of Maryland, College Park, MD.

12.

Lin

I-Fen

Schaeffer

Nora Cate

. 1995. “Using Survey Participants to Estimate the Impact of Nonparticipation.” Public Opinion Quarterly 59:236–58.

13.

Lynn

Peter

Clarke

Paul

. 2002. “Separating Refusal Bias and Non-contact Bias: Evidence from UK National Surveys.” Journal of the Royal Statistical Society Series D (The Statistician) 51:319–33.

14.

Michaud

Pierre-Carl

Kapteyn

Arie

Smith

James P.

Soest

Arthur van

. 2011. “Temporary and Permanent Unit Non-response in Follow-up Interviews of the Health and Retirement Study.” Longitudinal and Life Course Studies 2:145–69.

15.

Müller

Bettina

Castiglioni

Laura

. 2015. “Stable Relationships, Stable Participation? The Effects of Partnership Dissolution and Changes in Relationship Stability on Attrition in a Relationship and Family Panel.” Survey Research Methods 9:205–19.

16.

Roberts

Caroline

Vandenplas

Caroline

Stähli

Michèle Ernst

. 2014. “Evaluating the Impact of Response Enhancement Methods on the Risk of Nonresponse Bias and Survey Costs.” Survey Research Methods 8:67–80.

17.

Schouten

Barry

Cobben

Fannie

Bethlehem

Jelke

. 2009. “Indicators for the Representativeness of Survey Response.” Survey Methodology 35:101–13.

18.

Schouten

Barry

Bethlehem

Jelke

Beullens

Koen

Kleven

Øyvin

Loosveldt

Geert

Luiten

Annemieke

Rutar

Katja

Shlomo

Natalie

Skinner

Chris

. 2012. “Evaluating, Comparing, Monitoring, and Improving Representativeness of Survey Response through R-indicators and Partial R-indicators.” International Statistical Review 80:382–99.

19.

Smith

Tom W.

1984. “Estimating Nonresponse Bias with Temporary Refusals.” Sociological Perspectives 27:473–89.

20.

Stoop

Ineke A.L.

2005. The Hunt for the Last Respondent: Nonresponse in Sample Surveys. The Hague, the Netherlands: Netherlands Social and Cultural Planning Office.

21.

Trappmann

Mark

Gramlich

Tobias

Mosthaf

Alexander

. 2015. “The Effect of Events between Waves on Panel Attrition.” Survey Research Methods 9:31–43.

22.

Voorpostel

Marieke

. 2010. “Attrition Patterns in the Swiss Household Panel: An Analysis of Demographic Characteristics and Social Involvement.” Swiss Journal of Sociology 36:359–77.

23.

Voorpostel

Marieke

Lipps

Oliver

. 2011. “Attrition in the Swiss Household Panel: Is Change Associated with Drop-out?” Journal of Official Statistics 27:301–18.

24.

Watson

Nicole

Wooden

Mark

. 2014. “Re-engaging with Survey Non-respondents: Evidence from Three Household Panels.” Journal of the Royal Statistical Society Series A 177:499–522.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.09 MB