Using Online and Paper Surveys

Abstract

To examine the effectiveness of mixed-mode methodology, this article draws on a study that used both online and paper surveys to collect data on a sample of the over 50 population in a small New England town. Respondents were offered the option of responding online or with a paper survey, an approach that addresses some of the major concerns about using online surveys among groups with limited Internet access while also reducing costs and simplifying data entry. The authors demonstrate that response mode does not have a significant effect on survey answers once demographic variables (such as employment and income) are added to the analysis. In addition, the results from this survey suggest that the Internet is useful as a tool to collect data from those over 50.

Keywords

seniors older population senior center online survey web-based survey mixed mode

Online surveys often target populations expected to have regular access to the Internet, such as students or university staff (Best et al. 2001; Börkan 2010; Couper, Traugott, and Lamias 2001; Sax et al. 2003). Mixed-mode methods for surveys are increasingly popular (Dillman 2007), and there is some indication that there is a limited “mode effect” for online versus paper surveys (Börkan 2010). Despite this, few studies consider the effectiveness of online survey options specifically for older populations. For example, Shih and Fan’s (2008) meta-analysis of the response rate for 39 studies compared the effectiveness of online and paper surveys, not one of which specifically considered older (50+) populations.

Many researchers assume that the Internet is an ineffective way to access populations over the age of 50, at least in part because older populations are less likely to have access to and make use of online resources (Loges and Jung 2001; National Telecommunications and Information Administration [NTIA] 2002). But few analyses confirm these assumptions, and recent research suggests that a large portion of older adults use the Internet (Pew Internet & American Life Project 2011).¹

In this article, we examine the approach of combining online and paper surveys for effective data collection for those over the age of 50. Drawing on data from a survey of a small New England town, we compare the response rate and demographic patterns for online and paper surveys for those who are 50 and older.

Purpose

The original purpose for this survey was to assess the quality and rate of use for programs and services at the local senior center. When respondents were given the choice to complete a survey online or in paper form, we suspected that there would be some demographic differences between the two self-selected groups (i.e., income and age may be related to Internet access and facility). Based on other evaluations of mixed-mode survey design (Best et al. 2001; Börkan 2010; Sax et al. 2003; Dillman 2007) as well as online surveys of young cohorts and preexisting groups (Couper et al. 2001; Couper et al. 2007), we hypothesized that older adults’ demographic characteristics would outweigh response mode as important factors in understanding their lifestyles (e.g., health and well-being, mobility, contact with family, and use of local services for seniors). If response mode did indeed have a limited impact on the analytic outcomes of the survey, we could strengthen the argument that the mixed-mode approach is an appropriate way to increase response options for data collection among an older population.

Relevant Literature

Older populations and the Internet

There are striking differences in the use of digital technologies between older and younger populations, with older populations less likely to access the Internet (Loges and Jung 2001; NTIA 2002). And yet as younger and more technologically savvy populations age, knowing how to effectively survey the senior population will become more useful. Younger cohorts, particularly those under 65, are more likely to use the Internet than older cohorts (Eriksson-Backa 2003; Millward 2003), at least in part because they may still be in the workplace (Haddon 2000). In addition to age, demographic factors such as higher income and education (Loges and Jung 2001; Lorence and Park 2007; Morris, Goodman, and Brading 2007) are positively correlated with Internet use.

As for other concerns for older adults, such as frequency of leaving the house, it is difficult to parse whether Internet use prompts individuals to be more engaged with their communities or if those who are more connected with others are likely to feel more confident about using the Internet (Cody et al. 1999; Czaja et al. 1993; Czaja and Sharit 1998). For example, those over the age of 55 who do use the Internet report that Internet access has a positive impact on their lives (Morris et al. 2007), and there is some evidence that Internet access may help older individuals maintain their physical health (Philbeck 1997). White et al. (2002) found that older Internet users were less lonely and experienced less depression than their nonuser counterparts, though, again, causality is difficult to establish.

While there have been numerous studies on the relationship between Internet use and correlated concerns among older adults, Couper et al. (2007) conducted one of the few web-based analyses on respondents age 50 and over. Using a self-selected group, Couper et al. found that those older persons who use the Internet regularly are likely to be in better health, to have a higher income, and to be more educated than those who do not. Couper et al., however, conducted only a web-based survey and, perhaps more importantly, collected survey responses only from a subset of already willing participants. This study contributes to analyses of Internet use for data collection with a population over the age of 50, focusing on whether or not the response mode (online vs. paper) that respondents selected resulted in significantly different answers to key survey questions.

Senior centers and health assessment research

There are approximately 11,000 senior centers in the United States (National Council on Aging 2012), a significant drop since 2005 (Salari, Brown, and Jacqueline 2006). Senior centers can reduce psychological stress (Farone, Fitzpatrick, and Tran 2005), provide seniors with important information (Tang and Lee 2011), and improve social networks (Aday, Kehoe, and Farney 2006). This is particularly true when senior center programs and services reflect the needs of the specific populations they serve (Aday et al. 2006; Farone et al. 2005; Yoon 2005; Yoon and Lee 2007). In addition, as baby boomers age, the needs of the senior population are likely to shift (Milner 2007; Young 2005), and the ability of senior centers to recognize and meet these needs is important for their longevity as organizations. Despite this, many senior centers face reduced funding (Alt 1998; Hostetler 2011) and have a need to demonstrate their usefulness to public and private funding sources (Hostetler 2011; Taylor 2011).

Those who conduct health assessment surveys of senior populations often use senior centers as a convenient source for participants (some examples include Beverly et al. 2005; Choi and McDougall 2007; Fitzpatrick et al. 2005). While this is entirely understandable, those who use senior centers often differ from their non-center-using counterparts, and expanding the sampling method to draw from all seniors living in the community may increase the representativeness of the sample. Furthermore, community-wide health assessment endeavors may be improved by combining traditional paper survey methods with an online survey component. Adding the option for survey participants to respond online can increase accessibility by adding audio components (Dillman 2007), can simplify administration by utilizing skip fields and answer piping (Dillman 2007), and can reduce costs associated with paper surveys and data entry (Couper and Miller 2008; Wright 2006).

Online surveys and mixed methodology

The use of the Internet for collecting data is becoming more common (Couper et al. 2007). Marketing researchers began using web-based surveys before those in many other fields (Dolnicar, Laesser, and Matus 2009), though at this point the use of web surveys is common in any number of academic fields, including the social sciences, medicine, and education (Lefever, Dal, and Matthíasdóttir 2007; Shih and Fan 2008). Examining the efficacy of web-based surveys (Berrens et al. 2003; Best et al. 2001; Ilieva, Baron, and Healey 2002) is vital as it is clear this method of data collection is here to stay. Some studies have used the Internet to access difficult populations (Andrews, Nonnecke, and Preece 2003) such as egg donors (Curtis 2010) or to address sensitive topics such as how gay, lesbian, and bisexual college students define “having sex” (Mustanski 2001). As a result, it is important to examine the validity of web surveys more generally but also to consider how researchers can use the Internet to access underrepresented populations.

Because many studies that use online methods recruit participants from online sources, the most common critique leveled at the use of online surveys is the difficulty in achieving a representative sample, primarily because only a subset of the population has regular access to the Internet. Limited access to the Internet can include having an email address that works, the resources to go online, and the ability to use the Internet. We argue that online surveys can be used more effectively with populations that have limited access to the Internet if participants are recruited using paper methods and provided with an option to respond online because this allows researchers to draw from a more representative participant pool. This mixed-mode approach can help to address the most common issue with online survey use.

In an assessment of studies done using web-based surveys, Shih and Fan (2008) found that the average response rate for a web survey was 34% and that web-based surveys had about a 10% lower response rate than mailed surveys (p. 257). Research suggests that web-based surveys are most effective when used as part of a mixed-mode study (Börkan 2010; Couper et al. 2007; Greenlaw and Brown-Welty 2009; Schonlau, Asch, and Du 2003). While a mixed-mode methodology does not address all of the potential concerns for the use of online surveys, this approach does seem to provide higher response rates (Dillman et al. 2009; Ilieva et al. 2002), and for that reason mixed-mode surveys are worth considering even though they are more expensive than web-only surveys (Greenlaw and Brown-Welty 2009). Combining the use of paper and web surveys may also contribute to the robustness of outcomes (Dolnicar et al. 2009). This is particularly important when the outcomes of interest among the target population may be correlated to Internet use (Robinson and Martin 2009; Sum et al. 2008; Sum et al. 2009).

Survey Method

We examined data from a mixed-mode community-based survey of adults aged 50 and over, where respondents were offered the option to complete an online questionnaire or a mailed paper questionnaire. The primary goal of this survey was to assess the needs of the senior and soon-to-be senior population of the town of Amherst, Massachusetts. The Amherst Council on Aging (ACA) requested that the principal investigators (PIs) design and implement a survey to accomplish this goal. The survey instrument asked respondents about their use of the Amherst Senior Center (ASC), financial strains, health needs, and worries for the future. Respondents were not offered a gift or reward for participating in the survey.

The survey was mailed to individuals age 50 and older residing in Amherst. The ACA provided the PIs with a list created from the town’s street directory and the newsletter mailing list. The list included both those who rented and those who owned their home. The street directory was publicly available and linked to owning a landline, so those who did not have a phone or who used cell phones exclusively were not included. However, a recent study using the Current Population Survey indicated that the percentage of persons over 55 who had only a cell phone was less than 2% (Tucker, Brick, and Meekins 2007). An additional limitation was that individuals living in group quarters (such as nursing homes) were not included in the street directory. That said, the ASC mailing list included names that were not listed in the street directory, and so captured some of those who did not have a landline or who lived in group homes, resulting in a list that came as close as possible to a complete universe of residents age 50 and older living independently.

To address the potential weaknesses in using an online survey, the researchers in this study used a different approach than other online survey studies. Rather than relying on respondents having (and checking) a working email address, the researchers sent potential respondents a postcard inviting them to participate in a survey by giving them a link to participate online and letting them know that a paper copy of the survey would arrive in a couple of weeks for those who did not respond online. Identification numbers were assigned as passwords to enter the online survey and were used to track respondents and ensure that only those who did not respond online received a paper survey, The questions on the web version of the survey were identical and presented in the same order as the paper survey. Because the PIs collected both paper surveys and online surveys, and since lack of Internet access was not a barrier to participation, we can compare online and paper respondents and examine meaningful differences in their responses and assess online surveys as a resource for studying older populations.

The PIs randomly selected 1,800 names from the list (a random sample of approximately 30% of the total list) and contacted the potential respondents in two waves. Anticipating a response rate of 40% (Dillman 2007; Shih and Fan 2008), the 1,800 names represented a likely response pool of 720, which would allow for a reasonable assessment at the desired level of precision for a town the size of Amherst (Iarossi 2006).

The first wave of respondents (n = 1,100) was sent a postcard with a link to the survey and a password and identification number.² After two weeks, everyone in the first wave who had not yet filled out the survey online was sent a paper copy of the survey (similar to Millar and Dillman 2011). The mailing included a cover letter with further details, a return postage-paid envelope, and a paper questionnaire. The second wave of survey invitations (n = 700) was sent out four weeks after the initial mailing following the same procedure (postcard, then paper questionnaire).

Of the 1,800 survey invitations sent out, 118 were returned undeliverable, usually because the respondent had moved or was deceased. The research team received 735 usable survey responses (response rate = 43.7%), 212 online and 523 paper responses. Nine respondents (or 4.3% of the online respondents) returned a hard copy of the survey that included a letter or note indicating they had also filled the survey out online. Four of the nine were from the first wave, and the remaining five were from the second wave of surveys. The PIs opted to use the electronic version of these surveys because the data entry for the online survey was already completed. A comparison of respondents age 50 to 64 versus 65 and older showed that while the raw numbers of online respondents were about evenly distributed between the two age groups (n = 104, n = 108), a significantly higher proportion of respondents ages 50 to 64 selected the online survey, compared to the proportion of respondents ages 65 and older (33.9 vs. 25.2%, p = .007).

There were no follow-up mailings after the paper questionnaire was sent; although the response rate was good, additional funds for follow-up contact may have increased the response rate further. For the analyses reported in this article, we selected only responses where there were no missing values on the items of interest (n = 569, 139 online, 430 paper).³

Survey instrument

Several areas of program evaluation and needs assessment were included in the survey that covered respondents’ relationship to the senior center and their mobility and independence. For this study, to assess whether the response mode had an effect on the findings, we focused on the central themes of the survey: respondents’ independence and well-being and their use of ASC services.

Online versus paper

Responses were coded during data entry if they were returned online or on paper. Respondent codes ensured only one response mode.

Demographics

Standard demographic items included highest year finished in school, income, age, working full- or part-time, marital status (binary married or living with partner versus not married), gender (binary male or female), and race (binary White or non-White)

Health and well-being

We selected a few key items to approximate respondents’ emotional, physical, and financial health. Emotional health was measured using a single item, “Do you often feel sad or lonely?” where 1 = yes. Physical health was measured using a single item, “How would you rate your health,” and recoded so that 0 = very poor and 3 = excellent. Financial health combined 11 instances where respondents indicated whether they were concerned they could not afford important items (e.g., rent/mortgage, food, medicine) and constituted a scale ranging from 0 (not concerned about any items) to 1 (concerned about all 11 items).

Mobility and family contact

Responses to the question “How many days per week, on average, do you leave home for any reason?” were recoded so that 0 = none and 4 = every day. The survey included two questions about respondents contact with their family: “How often do you speak to members of your family on the phone or exchange emails?” and “How often do you see members of your family?” Responses were combined so that 0 = rarely or never have contact with family and 3 = daily contact with family.

Senior center services

The community-based survey asked seniors about their experiences with the ASC, including whether they used any services (referencing a list of 13 specific services and a general “other” category), which services they used, how frequently, and how satisfied they were with those services. Ever using any of the services (including health and fitness services, meal service, activities and trips, and educational services) was combined into a single item ranging from 0 = 0 services to 1 = used all services listed. Frequency of using services ranged from 0 = never used any services to 4 = used any service at least once a week.

Results

Our analyses compare online and paper respondents to examine any differences between the groups on central items of interest for the local senior center. Our analyses examine differences between respondents that are more significant than response mode (i.e., education, income, work status, and age).

When comparing outcomes of interest to the senior center, initial comparisons between online responders and paper responders suggested some differences. For example, online responders were less likely to have taken advantage of senior center programs, and they left the house on more days during an average week. To further investigate the possible effects of self-selected response mode, we used ordinary least squares (OLS) regressions to test issues of interest to the senior center. We found that any possible effects the online or paper response may have had on the survey outcomes were mediated by demographic differences between groups. Demographic differences that were more significant than response mode varied by dependent variable, and included education, income, work status, and age.

To address selection effects often present in nonexperimental social research and further complicated by the fact that respondents were not randomly assigned to a response mode, we included the Heckman two-step correction in the analyses, which allows us to account for possible selection bias to the extent that it may affect outcomes of interest (Bushway, Johnson, and Slocum 2007; Copas and Li 1997). In the first step, we used logistic regression to construct a probit estimate (Lee 1983; Smits 2001). The independent variables were demographic items: education, income, working full- or part-time, age, marital status (binary married or living with partner versus not married), gender (binary male or female), and race (binary White or non-White). This created the individual probabilities that were used to construct the individual probit scores, which were then used to compute the lambda for each case. In the second step, selected demographic variables used in constructing the probit estimate were excluded from the OLS regression models presented in this article (marital status, gender, and race), while other demographic variables were left in (education, income, work status, and age).

Response mode

Demographic differences between online and paper respondents are shown in Table 1. Online respondents, on average, reported having slightly more education than paper respondents, had a slightly higher income, were more likely to work full- or part-time, and were about two years younger.

Table 1.

Comparisons of Online and Paper Respondents.

n ^a	Online 139	Paper 430	Total 569
Demographics
Education (mean, range 1 = 8th grade or less to 6 = master’s or higher)	5.58	5.37*	5.42
Income (mean, range 1 = $8k or less to 8 = $100k or more)	6.46	6.11^†	6.20
Work full- or part-time (%)	49.6	37.0**	40.1
Age (mean, range 50 and over)	66.30	68.32*	67.83
Health and well-being
Emotional: Often feel sad or lonely	11.5%	11.4%	11.4%
Physical: Self-rated health (mean, range 1 = very poor to 4 = excellent)	3.35	3.35	3.35
Financial: Concerned they cannot afford any of 11 basic items, such as rent/mortgage, food, and medicine (mean, scale 0 = concerned about 0 items to 1 = concerned about all 11 items)	0.11	0.14	0.13
Mobility and family contact
Number of days in a week respondents leaves their house (mean, range 0 = 0 days to 4 = every day)	3.55	3.38*	3.42
Contact with family members (in person, phone, or email) (mean, range 0 = rarely or never to 3 = daily)	2.35	2.42	2.40
Senior center services
Ever used any of 14 senior center services (mean, range 0 = 0 services to 1 = all 14 services)	0.10	0.12	0.12
Frequency of using any senior center services (mean, range 0 = never to 4 = more than once a week)	0.87	1.04	1.00

To avoid missing values, only those who reported responses for every item in the analyses were included. The individual differences between online and paper responders shown here for this subsample (n = 569) were similar for the entire set of respondents (n = 735).

†

p ≤ .10. *p ≤ .05. **p ≤ .01.

We expected to see some differences in health and well-being, independence, and family contact when we compared online and paper respondents because we suspected that for this age group, respondents with ready access to technology and facility with the Internet might be more active or have more outside contact when compared with other seniors. As shown in Table 1, both online and paper responders reported low rates of feeling sad or lonely, rated their own health quite high, and were not concerned about many basic living expenses. Those who responded online left their house more frequently during an average week, but there were no differences between the groups in the frequency of their contact with family members. There were also no significant differences between online respondents in terms of whether or not they had ever used senior center services, and there were no significant differences in the frequency with which respondents used senior center services.

Having established that there were no significant differences on the substantive measures of interest to the ASC but that there were significant demographic differences between response modes, we devised OLS regression models to test the effect of response mode when accounting for demographic variables. We employed the Heckman two-step correction (lambda variable) to address selection bias that may be present as a result of self-selection to response mode. Finally, we constructed an alternative model to investigate additional factors, such as health and well-being, that could have a more significant effect than response mode on mobility, family contact, and use of the senior center, and also tested that model using the Heckman correction. Having established that the mixed-mode method is an appropriate way to survey the population, this last model is designed to consider alternative explanations for differences among older adults in a community-based needs assessment.

Health and well-being

In initial analyses, there were no apparent differences between online and paper respondents in terms of their self-reported emotional, physical, and financial health, and for the most part OLS regression analyses returned similar results. The only significant difference was on the emotional health item (often feeling sad or lonely), shown in Table 2. The inclusion of the lambda variable in the model (to account for self-selection bias between online and paper responders) affected the results so that online responders were significantly less likely to feel sad or lonely compared to paper responders (Model B).

Table 2.

Ordinary Least Squares Regressions on Respondents’ Health and Well-Being.

	Model A	Model B
Often feels sad or lonely
Constant	0.483***	0.480***
Responded online	0.030	−0.354*
Education	−0.135**	−0.101*
Income	−0.132**	−0.124**
Work full- or part-time	−0.044	−0.010
Age	0.011	0.007
Lambda		.394*
Adj. R²	.050	.058
Self-rated health
Constant	3.241***	3.241***
Responded online	−0.053	−0.053
Education	0.109**	0.109*
Income	0.240***	0.240***
Work full- or part-time	0.093*	0.093*
Age	−0.189***	−0.189***
Lambda		.000***
Adj. R²	.181	.179
Concerned about affording living expenses
Constant	0.704***	0.705***
Responded online	−0.042	0.122
Education	−0.088*	−0.102*
Income	−0.315***	−0.319***
Work full- or part-time	−0.022	−0.036
Age	−0.165***	−0.163***
Lambda		−.168
Adj. R²	.132	.132

Standardized coefficients.

p ≤ .05. **p ≤ .01. ***p ≤ .001.

Mobility and family contact

Although descriptive analyses pointed to slight differences in respondents’ mobility (online respondents left the house more days in a week than paper respondents), OLS regression results showed that when key demographic variable were accounted for, response mode did not have any correlation. As shown in Table 3, respondents’ education, income, work status, and age were all far more important in understanding respondents’ mobility than the response mode for the survey (Model A). The addition of the lambda factor to account for selection bias did not change the explanatory power of the model, and it had only a modest impact on the effect of each of the significant factors (Model B).

Table 3.

Ordinary Least Squares Regressions on Respondents’ Mobility and Family Contact.

	Model A	Model B
Number of days a week they leave their house
Constant	3.716***	3.718***
Responded online	0.034	0.128
Education	0.086*	0.078^†
Income	0.142***	0.140***
Work full- or part-time	0.155***	0.147**
Age	−0.216***	−0.215***
Lambda		−.095
Adj. R²	.173	.172
Frequency of contact (in person, phone, email) with family members
Constant	2.465***	2.461***
Responded online	−0.045	−0.250
Education	−0.125**	−0.107*
Income	0.229***	0.233***
Work full- or part-time	−0.013	0.005
Age	−0.006	−0.009
Lambda		.210
Adj. R²	.036	.038

Standardized coefficients.

†

p ≤ .10. *p ≤ .05. **p ≤ .01. ***p ≤ .001.

Similar analyses were conducted for frequency of contact with family, with slightly different results, also shown in Table 3. Again, response mode had no significant relationship to frequency of family contact, and the model changed very little with the introduction of the Heckman correction. Education was significant and had a negative relationship to frequent family contact, and income was significant with a positive relationship to frequent family contact.

Use of ASC services

Ever using any ASC services and the frequency of using ASC services were examined employing the same series of OLS regression models, as shown in Table 4. We confirmed that response mode did not have any relationship to whether or not respondents reported using senior center services, and demographic variables were the strongest predictors of using ASC services. In all three models, education and age were significant and positively correlated with using ASC services, and income and working full- or part-time were significant and negatively correlated. The introduction of the Heckman correction had no notable effect on the analyses for using ASC services. The demographic variables were all significant in the health and well-being models.

Table 4.

Ordinary Least Squares Regressions on Respondents’ Use of Senior Center Services.

	Model A	Model B
Ever Used Senior Center Services
Constant	−0.112^†	−0.113
Responded online	0.012	−0.098
Education	0.103**	0.113**
Income	−0.298***	−0.295***
Work full- or part-time	−0.176***	−0.166***
Age	0.274***	0.273***
Lambda		.113
Adj. R²	.273	.273
Frequency of Using Senior Center Services
Constant	−0.559	−0.563
Responded online	0.002	−0.117
Education	0.086*	0.097*
Income	−0.278***	−0.275***
Work full- or part-time	−0.213***	−0.203***
Age	0.266***	0.265***
Lambda		.121
Adj. R²	.286	.286

Standardized coefficients.

†

p ≤ .10. *p ≤ .05. **p ≤ .01. ***p ≤ .001.

Discussion

The results discussed above indicate that there were few differences between the online and paper respondents, suggesting than an online option for surveys is a reasonable strategy for researchers seeking to access older populations, including seniors ages 65 and over. The small demographic differences were expected (online respondents had a little more education and a little more income, were more likely to work, and were a little younger) and were consistent with other studies that found positive correlations between higher income and education and increased Internet use. We used the Heckman two-step correction to account for those differences between groups within the survey and found that the introduction of the resulting factor did little to change the OLS regression models that were constructed without it. There is evidence of few differences between online and paper respondents to surveys of younger populations (Carini et al. 2003), and this study confirms that the same is true for older populations.

We expected that demographic differences would be more important than response mode in predicting respondents’ mobility, family contact, and use of the ASC services. We found that when it came to issues of interest in a community survey of senior citizens, demographic factors—such as higher education or employment status—far outweighed the differences between the two self-selected response modes. That is, response mode did not account for the most significant variations among respondents. There was one result (Table 2, Model B) where once we accounted for self-selection, online responders were less likely to feel sad or lonely than paper responders, a finding consistent with literature linking Internet use to emotional health among older adults (White et al. 2002).

The Internet’s value as a research tool is undeniable; and yet it is important for researchers to examine the best ways to use the Internet to effectively collect data. Most studies comparing online and paper respondents are designed by randomly assigning participants to a response group rather than allowing respondents to self-select their response mode. Self-selected response mode alone did not have any significant effects on the issues of interest for the senior center survey. But the demographic factors that were significantly different between the online and paper respondents (i.e., age, work status) were also significantly correlated with the issues of interest. These findings suggest that by giving respondents an option for how to complete a survey, mixed-mode survey administration may still reach a broad population, and the self-selected mode itself has little influence on the major outcomes of interest.

Furthermore, given that a large percentage of the population is increasingly comfortable with technology, as more computer-savvy generations age it is increasingly feasible to conduct online surveys with older populations. Our findings suggest that researchers may benefit from including online surveys as part of their work when they are collecting data from those 50 and over. Where practical, researchers may also consider oversampling nonworking, older, less educated respondents with lower incomes as one way to address discrepancies between the likely online response pool and the overall population.

In addition, our data collection methods used a sample that drew from seniors living in the community, supplemented with but not wholly drawn from lists provided by the local senior center and/our Council on Aging. While some studies of older populations used convenience samples—conducting online surveys with groups of seniors who were already online (Wu and Ozok 2009) or needs assessments with seniors who were already participating in senior programming (Beverly et al. 2005)—this sampling method expanded the response pool so that the ASC could potentially learn from a cross section of older adults living in their community, not just older adults who were already part of an online community or on the senior center mailing list. The findings in this article strengthen the argument that offering older adults the option to respond online can assist senior centers in improving services to their community and increase recruitment for health assessment surveys of senior populations.

One limitation to this study is that the population surveyed was more highly educated when compared with the general U.S. senior population. Given that the study occurred in a college town and did not extensively survey those who lived in institutional settings such as nursing homes, the level of education in the respondent population is not surprising, and it limits the generalizability of the findings. Furthermore, online surveys do not appear to redress a major issue in all data collection: Less educated individuals are less inclined to fill out surveys (Couper et al. 2007; Dillman et al. 2009). Mixed-mode survey administration may not address this concern over nonresponse that is common to survey research. Related to this, there were no opportunities to compare respondents to nonrespondents, as the lists were drawn from a community list that did not provide data on nonrespondents; this is a common limitation to community-based research. The sample included for these analyses was not only highly educated but also highly responsive, as they were able to answer (positively or negatively) all of the items that were used to construct our models, and so another limitation was our choice to use listwise elimination for missing values rather than imputing missing data. While the choice was appropriate given the uncertainty of nonresponse and hazards of imputation, this introduced bias toward respondents who are more comfortable completing such a questionnaire, a bias perhaps consistent with the high level of education of the possible response pool. Finally, two thirds of respondents still chose to complete paper (71% of respondents) rather than online (29%) questionnaires, so while online surveys are useful with this population, we would not say that older adults are universally likely to respond to online surveys.

Conclusion

Factors other than response mode explain the variations in the results between online and paper surveys. For senior center administrators and those conducting community-wide health assessments, the findings from this study suggest that a mixed-mode approach can be used to increase efficiency and reduce costs (paper, postage, data entry) associated with mailed surveys and should be considered for use in large-scale data collection among an older population, including a senior population ages 65 and older.

Footnotes

Acknowledgements

Thanks to Professors Randall Stokes and Doug Anderton for their support.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: These data were available through a community research study funded by the Amherst Senior Center and the Amherst Council on Aging.

Notes

Author Biographies

Dana Huyser de Bernardo is a PhD candidate. Her research areas address religious and political affiliation and identity. Her applied research experience focuses on public health, including community needs assessment and program evaluation.

Anna Curtis is a PhD candidate in residence. She is interested in punishment and corrections, law and society, gender and family, as well as qualitative methodology. Her research areas include masculinity, fatherhood, and incarceration as well as reproductive technology.

References

Aday

Kehoe

Farney

2006. “Impact of Senior Center Friendships on Aging Women Who Live Alone.” Journal of Women and Aging 18:57-73.

Alt

1998. “Future Directions for Public Senior Services: Meeting Diverging Needs.” Generations 22:29-33.

Andrews

Nonnecke

Preece

2003. “Electronic Survey Methodology: A Case Study of Reaching Hard-to-Involve Internet Users.” International Journal of Human-Computer Interaction 16:185-210.

Berrens

Bohara

Jenkins-Smith

Silva

C. I.

Weimer

2003. “The Advent of Internet Surveys for Political Research: A Comparison of Telephone and Internet Samples.” Political Analysis 11:1-22.

Best

Krueger

Hubbard

Smith

2001. “An Assessment of the Generalizability of Internet Surveys.” Social Science Computer Review 19:131-45.

Beverly

Mcatee

Costello

Chernoff

Casteel

2005. “Needs Assessment of Rural Communities: A Focus on Older Adults.” Journal of Community Health 30:197-212.

Börkan

2010. “The Mode Effect in Mixed-Mode Surveys: Mail and Web Surveys.” Social Science Computer Review 28:371-80.

Bushway

Johnson

B. D.

Slocum

L. A.

2007. “Is the Magic Still There? The Use of the Heckman Two-Step Correction for Selection Bias in Criminology.” Journal of Quantitative Criminology 23:151-78.

Carini

R. M.

Hayek

J. C.

G. D.

Kennedy

J. M.

Ouimet

J. A.

2003. “College Student Responses to Web and Paper Surveys: Does Mode Matter?” Research in Higher Education 44:1-19.

10.

Choi

McDougall

2007. “Comparison of Depressive Symptoms Between Homebound Older Adults and Ambulatory Older Adults.” Aging and Mental Health 11:310-22.

11.

Cody

M. J.

Dunn

Hoppin

Wendt

1999. “Silver Surfers: Training and Evaluating Internet Use Among Older Adult Learners.” Communication Education 48:269-86.

12.

Copas

J. B.

H. G.

1997. “Inference for Non-Random Samples.” Journal of The Royal Statistical Society 59:55-95.

13.

Couper

Kapteyn

Schonlau

Winter

2007. “Noncoverage and Nonresponse in an Internet Survey.” Social Science Research 36:131-48.

14.

Couper

M. P.

Miller

P. V.

2008. “Web Survey Methods: Introduction.” Public Opinion Quarterly 72:831-35.

15.

Couper

Traugott

Lamias

2001. “Web Survey Design and Administration.” Public Opinion Quarterly 65:230-53.

16.

Curtis

2010. “Giving ‘til It Hurts: Egg Donation and the Costs of Altruism.” Feminist Formations 22:80-100.

17.

Czaja

Guerrier

Nair

Landauer

1993. “Computer Communication as an Aid to Independence for Older Adults.” Behaviour and Information Technology 12:197-207.

18.

Czaja

Sharit

1998. “Age Differences in Attitudes Toward Computers.” Journals of Gerontology, Series B: Psychological Sciences 53B:P329-40.

19.

Dillman

2007. Mail and Internet Surveys: The Tailored Design Method. 2d ed. Hoboken, NJ: John Wiley.

20.

Dillman

Phelps

Tortora

Swift

Kohrell

Berck

Messer

2009. “Response Rate and Measurement Differences in Mixed-Mode Surveys Using Mail, Telephone, Interactive Voice Response (IVR) and the Internet.” Social Science Research 38:1-18.

21.

Dolnicar

Laesser

Matus

2009. “Online Versus Paper: Format Effects in Tourism Surveys.” Journal of Travel Research 47:295-316.

22.

Eriksson-Backa

2003. “Who Uses the Web as a Health Information Source?” Health Informatics Journal 9:93-101.

23.

Farone

Fitzpatrick

Tran

2005. “Use of Senior Centers as a Moderator of Stress Related Distress Among Latino Elders.” Journal of Gerontological Social Work 46:65-83.

24.

Fitzpatrick

Gitelson

Andereck

Mesbur

E. S.

2005. “Social Supports Factors and Health Among a Senior Center Population in Southern Ontario, Canada.” Social Work in Health Care 40:15-37.

25.

Greenlaw

Brown-Welty

2009. “A Comparison of Web-Based and Paper-Based Survey Methods: Testing Assumptions of Survey Mode and Response Cost.” Evaluation Review 33:464-80.

26.

Haddon

2000. “Social Exclusion and Information and Communication Technologies: Lessons from Studies of Single Parents and the Young Elderly.” New Media and Society 2:387-406.

27.

Hostetler

A. J.

2011. “Senior Centers in the Era of the ‘Third Age:’ Country Clubs, Community Centers, or Something Else?” Journal of Aging Studies 25:166-76.

28.

Iarossi

2006. The Power of Survey Design: A User’s Guide for Managing Surveys, Interpreting Results, and Influencing Respondents. Washington, DC: World Bank.

29.

Ilieva

Baron

Healey

2002. “Online Surveys in Marketing Research: Pros and Cons.” International Journal of Market Research 44:362-80.

30.

Lee

L. F.

1983. “Generalized Econometric Models With Selectivity.” Econometrica 51:507-13.

31.

Lefever

Dal

Matthíasdóttir

Á.

2007. “Online Data Collection in Academic Research: Advantages and Limitations.” British Journal of Educational Technology 38:574-82.

32.

Loges

Jung

2001. “Exploring the Digital Divide: Internet Connectedness and Age.” Communication Research 28:536-62.

33.

Lorence

Park

2007. “Group Disparities and Health Information: A Study of Online Access for the Underserved.” Health Informatics Journal 14:29-38.

34.

Millar

M. M.

Dillman

D. A.

2011. “Improving Response to Web and Mixed-Mode Surveys.” Public Opinion Quarterly 75:249-69.

35.

Millward

2003. “The ‘Grey Digital Divide’: Perception, Exclusion and Barriers of Access to the Internet for Older People.” First Monday 8. http://www.firstmonday.org/htbin/cgiwrap/bin/ojs/index.php/fm/article/view/1066/986

36.

Milner

2007. “Recreation and the Age Wave.” Journal on Active Aging 6:72-77.

37.

Morris

Goodman

Brading

2007. “Internet Use and Non-Use: Views of Older Users.” Universal Access in the Information Society 6:43-57.

38.

Mustanski

2001. “Getting Wired: Exploiting the Internet for the Collection of Valid Sexuality Data.” Journal of Sex Research 38:292-301.

39.

National Council on Aging. 2012. “Senior Centers: Fact Sheet.” http://www.ncoa .org/press-room/fact-sheets/senior-centers-fact-sheet.html.

40.

National Telecommunications and Information Administration (NTIA). 2002. “A Nation Online: How Americans Are Expanding Their Use of the Internet.” http://www.eric.ed.gov/PDFS/ED462928.pdf

41.

Pew Internet & American Life Project. 2011. “Demographics of Internet Users.” http://www.pewinternet.org/Static-Pages/Trend-Data/Whos-Online.aspx.

42.

Philbeck

1997. “Seniors and the Internet.” Cybersociology Magazine 2. http://www.cybersociology.com/files/2_2_philbeck.html.

43.

Robinson

Martin

2009. “Social Attitude Differences Between Internet Users and Non-Users: Evidence from a General Social Survey.” Information, Communication and Society 12:508-24.

44.

Salari

Brown

Jacqueline

2006. “Conflicts, Friendship Cliques and Territorial Displays in Senior Center Environments.” Journal of Aging Studies 20:237-52.

45.

Sax

Gilmartin

Bryant

2003. “Assessing Response Rates and Nonresponse Bias in Web and Paper Surveys.” Research in Higher Education 44:409-32.

46.

Schonlau

Asch

2003. “Web Surveys as Part of a Mixed-Mode Strategy for Populations That Cannot Be Contacted by E-mail.” Social Science Computer Review 21:218-22.

47.

Shih

Fan

2008. “Comparing Response Rates From Web and Mail Surveys: A Meta Analysis.” Field Methods 20:249-71.

48.

Smits

2001. “Performing the Heckman Two-Step Procedure to Control for Selection Bias With SPSS.” http://home.planet.nl/~smit9354/selbias/selbias.html.

49.

Sum

Mathews

Hughes

Campbell

2008. “Internet Use and Loneliness in Older Adults.” Cyberpsychology and Behavior 11:208-11.

50.

Sum

Mathews

Pourhasem

Hughes

2009. “Internet Use as a Predictor of Sense of Community in Older People.” Cyberpsychology and Behavior 12:235-39.

51.

Tang

Lee

2011. “Social Support Networks and Expectations for Aging in Place and Moving.” Research on Aging 33:444-64.

52.

Taylor

2011. “An Historic Opportunity for Senior Centers.” http://www.ncoa.org/national-institute-of-senior-centers/nisc-news/an-historic-opportunity-for.html

53.

Tucker

Brick

Meekins

2007. “Household Telephone Service and Usage Patterns in the United States in 2004: Implications for Telephone Samples.” Public Opinion Quarterly 71:3-22.

54.

White

McConnell

Clipp

Branch

Sloane

Pieper

Fox

2002. “A Randomized Controlled Trial of the Psychosocial Impact of Providing Internet Training and Access to Older Adults.” Aging and Mental Health 6:213-21.

55.

Wright

K. B.

2006. “Researching Internet-Based Populations: Advantages and Disadvantages of Online Survey Research, Online Questionnaire Authoring Software Packages, and Web Survey Services.” Journal of Computer-Mediated Communication 10. http://jcmc.indiana.edu/vol10/issue3/wright.html

56.

Ozok

2009. “Medication Adherence Among the Elderly and Technology Aids: Results from an Online Survey Study.” Pp. 719-27 in Online Communities, LNCS 5621, edited by A. A. Ozok and P. Zaphiris. Berlin: Springer-Verlag.

57.

Yoon

Lee

2007. “Impact of Religiousness, Spirituality, and Social Support on Psychological Well-Being Among Older Adults in Rural Areas.” Journal of Gerontological Social Work 48:281-98.

58.

Yoon

S. M.

2005. “The Characteristics and Needs of Asian-American Grandparent Caregivers: A Study of Chinese-American and Korean-American Grandparents in New York City.” Journal of Gerontological Social Work 44:75-94.

59.

Young

2005. “Centers (Don’t Say Senior) Look to Get Hip.” Arizona Republic. http://www.azcentral.com/families/articles/1201fam_seniorcenter.html.