Abstract
This study focuses on two topics that have heretofore received little attention: ethical standards for publishing tourism research and assessment of the effectiveness of such standards in generating compliance. A random sample of research articles published by 75 tourism journals in 2013 was drawn and 45 were identified that employed the probability sample survey method. Ten rules for ethical reporting of survey results were derived from the World Association for Public Opinion Research Code of Ethics, designed to encourage transparency of method, and applied to the articles. Most articles failed to comply with most of these rules, documenting that the articles did not incorporate the value of research transparency, a foundation of inquiry in social sciences. Recommendations are presented on how to encourage future papers employing this method published in tourism journals to comply with ethical standards for disclosure in order to improve the transparency and credibility of tourism research.
Introduction
Ethical and moral considerations reside at the heart of tourism. MacCannell (2011, 228) argues that persons intentionally crossing moral boundaries defines them as tourists. Such transients are, by their nature, disruptive to host communities to greater or lesser extent. Maintaining order, he argues, necessarily requires visitor awareness of and adherence to the ethical standards of places visited: standards of oughtness distinguishing hosts’ agendas of right and just behavior from those of visitors.
Reflecting MacCannell’s view of ethics as characteristic of tourism, published studies of tourism ethics have predominantly addressed ethical standards in relation to the behavior of visitors and hosts. Indeed, there are a plethora of journal articles, book chapters, and entire books (e.g., Hall 1993; Smith and Duffy 2003; Fennell 2006; Fennell and Malloy 2007; MacCannell 2011; Lovelock and Lovelock 2013) devoted to ethical dimensions of the behavior of visitors and their hosts.
However, few studies have been published on the ethics of research in tourism. And what is extant on this topic addresses only selected issues of researcher interactions with visitors and hosts as subjects of study. Much that has been published on ethical behavior of visitors and hosts has addressed the need for and nature of codes of conduct that define ethical behavior in tourism and purpose to encourage it. But researchers active in the field of tourism ethics have complained that such codes have not been evaluated as to their effectiveness in producing desired behaviors (Malloy and Fennell 1998; Lovelock and Lovelock 2013). These research gaps—lack of investigation of codes of ethical conduct for tourism research and failure to evaluate the effectiveness of codes of ethical conduct in tourism domains—suggest that “ethical reflexivity” is lacking in tourism research (MacCannell 2012, 185).
Moscardo (2010, 206) presents evidence that there is “an increasing awareness of the need for more critical and reflective approaches to tourism research.” She further maintains that “tourism research is particularly susceptible to challenging and difficult ethical questions” (2010, 210). MacCannell argues that “a research domain without ethical reflexivity, that refuses to come to grips with itself and its subject matter in ethical terms, is doomed to failure” (2012, 185). Fennell (2013, 421) expands on this prospect: “a research domain that refuses to organize itself in a way that allows researchers to build on past work because of this formlessness might also be doomed to failure.” Here, Fennell alludes to the failure to develop new knowledge through research employing rigorous methods, permitting validation and encouraging extension of results, all essential to growing knowledge.
This study is designed to make a substantive methodological contribution to the tourism research literature by addressing these gaps in reflexivity by (1) deriving a set of ethical standards for publishing tourism research from a professional code of ethics designed to encourage research transparency and (2) applying a method for assessing the degree of compliance with these standards in published tourism research.
Briefly, this present study comprises the following steps.
Identify a large set of academic journals addressing tourism, hospitality, sport, event, meetings and conventions, and leisure subjects (herein, “tourism journals”).
Draw a probability sample of one issue from each of the journals for a selected year.
Researchers read every research article in the selected issues to identify those employing a specific research method of research—the probability sample survey.
Translate an internationally recognized professional code of ethics specifying ethical standards for publishing articles employing this survey method into a set of explicit binary rules of compliance or noncompliance.
Researchers read each article selected in step 3 and apply the binary rules to ascertain compliance with each ethical standard, with coding disagreements resolved by the author.
Summarize findings of compliance by addressing three research questions.
Analyze findings regarding the effectiveness of the code and recommend steps to improve compliance of future articles employing the probability sample survey method published in tourism journals.
Present recommendations for extending this process to other research methods used in tourism research articles.
Definitions
I adopt here the definition of ethics proposed by Fennell (2009, 213): “Defined, ethics is the rules, standards and principles that dictate right, good, and authentic conduct among members of a society or profession” (italics in original). This paper focuses on the ethics—rules of right conduct—among researchers employing probability sample surveys as the method to derive knowledge about defined populations. Such rules and principles are often codified in “professional codes of ethics,” defined here as “written, distinct and formal documents, issued by professional associations, that attempt to guide the professional behavior of their members” (Pater and Van Gils 2003, 765). Such codes are instrumental in directing the behavior of members of the profession and protecting stakeholders (e.g., clients, other researchers, the general public) from malfeasance (Lee and Tsang 2013). They are the preferred method of associations, business firms, nonprofit organizations, and governments for expressing tourism ethics (Fennell 2006; Fennell and Malloy, 2007, 9). In order to be effective, these codes must relate values of the organization or profession to member practice ( Tribe 2002) and become the habitual way of daily operations of their subjects (Fennell and Malloy 2007).
The “probability sample survey,” the methodological subject of this study, is defined as a systematic method of gathering information from a sample of units selected from a larger population in a way that every population unit has a known, nonzero chance of being selected, for the purpose of constructing quantitative descriptors of that larger population. (author, based on Dillman et al. 2009 and Groves et al. 2009)
This definition eliminates from consideration survey methodologies employing convenience samples, quota samples, cascade samples, respondent self-selection, or any other method that substitutes some selection method for random selection of respondents (Fowler 2009). None of these nonrandom sampling methods permit valid generalization of means, variances, or percentages from the samples to larger, defined populations (Neuendorf 2002, 87).
Literature Review
Prior Studies: Ethics and Tourism
Studies of the ethics of various aspects of tourism have focused on a number of areas, and there is not enough space to adequately discuss them here. Interested readers are directed to book-length treatments of the subject (e.g., Smith and Duffy 2003; Fennell 2006; Fennell and Malloy 2007; MacCannell 2011; Lovelock and Lovelock 2013).
Prior Studies: Ethics and Tourism Research
Despite the substantial number of published tourism ethics studies, there are relatively few articles on the applications of ethics to tourism research (Moscardo 2010). For example, Perdue (1991) examined the field of visitor surveys to determine the economic impact of tourists on a geographic area and presented a list of potential ethical problems. He maintained that presenting results from a convenience sample of visitors as being as valid as those derived from a probability sample is unethical behavior. McCleary (1994) discussed the need for a professional code of ethics in publishing tourism/hospitality research studies that addressed co-authorship credit for scholarship, multiple simultaneous submissions of manuscripts to journals, the number of articles submitted from a single study or database, complete explication of methods employed, and misrepresentation of methods. Ryan (2005) explored the duties of scientific inquiry in the field of tourism research and concluded that researchers must act with honesty and integrity while pursuing justice.
Bernstein and Friedman (2013) observed resident attitudes toward sport facilities and events as disguised observers during an election campaign in a US city. In the process, they encountered ethical issues when they were “discovered.” In the same vein, Mura (2015) discussed ethical issues he detected in ethnographic fieldwork, such as harm to the researcher and tensions of being an ethical researcher while studying unethical practices. Capistrano (2013) investigated the shared social interactions reported by hosts and their related guests in friends-and-relatives visits to a Filipino community in New Zealand. He struggled with the ethical issues of ensuring anonymity while dealing with sensitive topics connecting hosts and guests. Finally, Fennell (2013) explored ethical dimensions of university pressure on faculty to publish in top-rated tourism journals and its deleterious effects on academic freedom and knowledge growth.
This review finds that published articles on the ethics of tourism research thus far have focused on qualitative data-gathering processes. None of them have addressed the ethics of reporting on a distinct quantitative data collection method occasionally employed in research manuscripts published in tourism journals. The present study addresses this deficiency.
Research Design
Social science research focuses on human beings. There are two ethical domains that have generally been recognized for research on human subjects (Groves et al. 2009, 371). One deals with the protection of human subjects during the conduct of research (Mura 2015; Veal 2011; Sirakaya-Turk et al. 2011; Arendt et al. 2012), and the other domain comprises protection of clients and other audiences in the presentation of research findings, the focus of this article.
First Ethical Domain: Protection of Human Subjects
Beginning with the Nuremburg Trials of German military personnel and government officials accused of World War II war crimes, governments and other institutions in many countries have agreed that there be a set of firm principles that define ethical research regarding human subjects and that carefully limit potential negative effects (Arango 2015). The Declaration of Helsinki of Ethical Principles for Medical Research Involving Human Subjects initially adopted by the World Medical Association in 1964 (now in its seventh revision) declared that respect for individuals and their right to make informed decisions during such research is morally binding on physicians throughout the world (World Medical Association 2015).
Following these developments, many countries passed laws requiring ethical treatment of human subjects in medical and other social research (Veal 2011, chapter 4; Sirakaya-Turk et al. 2011; Bryman 2001, chapter 24). The covered research methods include surveys of human populations. The ethical principles applied include respect for persons, preventing harm to subjects, maximizing human benefits of such research, and equitable distribution of the benefits and burdens of the research. Fisher and Anushko (2008) have provided a particularly complete and lucid discussion of this first ethical domain of research, and the reader who seeks more background on the subject is directed to this resource.
Second Ethical Domain: Protection of Clients and Public
The other ethical domain relating to surveys of human subjects addresses the protection of clients for whom the research is conducted and of the public as the audience (Groves et al. 2009, 374). Clients and other audiences for probability sample survey research must be shielded from misconduct in these surveys, such as falsification of results, procedures that introduce bias, manipulation of processes designed to distort the results, and plagiarism. The tool for accomplishing such protection is a set of standards embodied in a code of ethics for probability sample surveys that advance the value of research transparency. Transparency of research method requires presenting the techniques used to collect and interpret survey information for public scrutiny that permits assessment, replication, and verification by other researchers, the foundation of scientific inquiry (Babbie 2007, 146, 334).
The essence of probability sample surveys is the generalization of means and percentages to a defined population from a sample of units selected from that population. Rather complex methodologies are employed to achieve this objective, methodologies that if misused, can bias statistics from the sample so that they do not accurately imply what the corresponding population parameters are. For example, the effective response rate in such a survey measures the percentage of the units in the defined sample that actually provide responses sought in a survey. If this response rate is low, this suggests that the collection of responses is substantially generated by respondents who found it convenient or rewarding to complete the survey. This is likely to bias the means and percentages generated by the sample away from the means and percentages of the population. Unless a researcher publishes how she or he defined the population, selected the respondents, gathered responses, encouraged cooperation through incentives, and achieved a certain response rate, the public cannot accurately assess whether the survey is valid, that is, produces statistics that can be generalized to the defined population without bias.
Moreover, unless details of the methodology are published, other researchers cannot fairly replicate the results of a given survey or duplicate the technique for additional populations (Freese 2007). Comparisons of results from two or more different surveys fail to accurately measure respondents’ differences if the methodologies differ, and this is only evident if complete methodologies of probability sample surveys are published (European Society for Opinion and Market Research 2014). Failures to publish complete methodologies for these surveys can mislead the public into believing reported results truly represent a larger population, or that two surveys taken at different times or from different populations can be validly compared. Moreover, such reporting failures prevent verification—the ability to replicate a completed research study to verify its findings—one of the foundations of sound social research (Bryman 2001, 30; Babbie 2007). Over the long run, such failures discourage public confidence in survey research, reducing the contributions this efficacious method can make to public understanding (European Society for Opinion and Market Research 2008).
The Professional Code of Ethics
Scholars investigating ethical domains have endorsed the proposition that the “accepted and expected” instrument for communicating rules, standards, and/or principles that ensure transparent and ethical research is the code of ethics (Fennell and Malloy 2007, 9). As noted above, I focus here on professional codes of ethics (rather than organizational or corporate codes) designed to encourage compliance of the members of a profession or discipline with certain ethical principles. Among these principles are the following (Moscardo 2010, 207–8):
Research integrity and quality, where it is argued that only valid, reliable, unbiased or good research is ethical;
Fidelity, truth and transparency which all refer to the trustworthiness of the researcher in their dealings with all those involved in the research including participants, other stakeholders and other researchers.
Such codes of ethics offer a number of benefits to those covered by it. Frankl (1989) identified six such benefits:
Enabling document. The code offers guidance to individuals and groups by simplifying the moral universe through a framework that allows for the evaluation of alternative courses of action.
Source of public evaluation. The code serves as a basis for the public’s expectation of professional performance, thus holding group members more accountable for their actions.
Professional socialization. The code helps to foster pride in the profession along with strengthening identity and allegiance of the membership.
Enhance the profession’s reputation and public trust. The adoption of a code of ethics allows the profession to gain the public’s trust and to enhance its own status. This leads to confidence and respect as well as increased social and economic rewards.
Deterrent to unethical behavior. Codes act to restrain unethical behavior through the threat of sanctions and by making it a duty to report errant colleagues. This plants the seeds of a monitoring system that should be observed by of all in preserving the integrity of the group.
Fennell and Malloy (2007, 24) add a seventh benefit: The code should act as a source of support for professionals against the erosion of power and improper demands made by bureaucrats or clients, or unreasonable demands by employers.
Fennell and Malloy (2007), in their expansive treatment of tourism codes of ethics, argue that professionals across a wide variety of fields legitimize themselves through such codes. The codes outline “group norms, values, expectations and responsibilities” in order to guide public expectations of professional performance, deter unethical behavior and enable assessment of compliance with the values stated in the code.” Fleckenstein and Huebsch (1999) agree, adding that such codes maintain harmony in a profession and help settle disputes among members.
Such codes can be used to assess compliance with these ethical standards and to determine whether an action or work product in the professional field or discipline is ethical or not. As Fennell and Malloy note (2007, 56), “Implicit in a code is the belief or hope that the outcome of adhering to the code is ethical behavior.”
The WAPOR Code
One formal professional code of ethics applicable to a number of published tourism studies is promulgated by the World Association for Public Opinion Research (WAPOR) as the WAPOR Code of Ethics (World Association for Public Opinion Research 2015). According to its constitution, WAPOR was founded in 1947 to a) promote in each country of the world the right to conduct and publish scientific research on what the people and its groups think and how this thinking is influenced by various factors, b) promote the knowledge and application of scientific methods in this objective, c) assist and promote the development and publication of public opinion research worldwide, d) promote international cooperation and exchange among academic and commercial researchers, journalists and political actors, as well as between the representatives of the different scientific disciplines. (World Association for Public Opinion Research 2015)
The WAPOR Code “defines professional ethics and practices in the field of public opinion research” and explains that these standards are promulgated in order
to advance the use of science in the field of public opinion research;
to protect the public from misrepresentation and exploitation in the name of research;
to maintain confidence that researchers in this field are bound by a set of sound and basic principles. (World Association for Public Opinion Research 2015, 1)
The core research methodology in public opinion research is the probability sample survey defined above. In addition to directly encouraging ethical publication of such survey results, the WAPOR Code of Ethics has been influential in the development of codes of ethics governing survey method disclosure published by the American Association for Public Opinion Research (AAPOR) in its “Code of Professional Ethics and Practices” (AAPOR 2015) and the European Society for Opinion and Market Research (ESOMAR) in its ESOMAR/WAPOR Guideline on Opinion Polls and Published Surveys (ESOMAR 2014). These codes are virtually identical to WAPOR’s code in terms of standards specified. Moreover, WAPOR is a member of the International Social Science Council (ISSC), an associate organization of UNESCO, and drafted Guidelines for Exit Polls and Election Forecasts for ISSC that reiterate the elements of WAPOR’s Code of Ethics. These relationships suggest that the ethical standards in the WAPOR Code of Ethics have been widely adopted by the world’s survey research profession.
Structure of the WAPOR Code
The WAPOR Code of Ethics “prescribes principles of ethical practices for the guidance of its members, and a framework of professional standards that should be acceptable to users of research and to the public at large” (World Association for Public Opinion Research 2015, 1). These principles reflect requirements of scientific investigation, and adherence to them helps maintain the public’s confidence in the researchers and the research they generate on public opinion. WAPOR Code Article II, section C, specifies 11 “Rules of Practice Regarding Reports and Survey Results,” stating, “Every complete report on a survey should contain an adequate explanation of the following relevant points” (page 3) listed verbatim in Figure 1.
WAPOR rules of practice regarding reports and survey results.
In sum, when publishing the results of probability sample surveys, researchers have ethical obligations toward their clients and the general public. These obligations are to disclose certain information about a survey plan and its conduct when the findings are released. WAPOR pursues this objective by requiring “that key information is published alongside a poll or survey report to ensure professional and transparent reporting so the audience has the opportunity of judging whether or not it agrees with the conclusions drawn from the research” (European Society for Opinion and Market Research 2014, 6).
Employing the WAPOR Code
I employ the WAPOR Code to assess the extent of compliance of articles utilizing the probability sample survey method published in tourism journals. Assessing “compliance” with codes of ethics includes auditing and verification (Kolk and van Tulder 2002, referenced in Fennell and Malloy 2007, 77). I accept Fennell and Malloy’s contention (2007, 15) that codes of conduct require explicit investigations of compliance and identification of instances of noncompliance, defined as behavior that does not conform to the prescriptions in a code of ethics.
Given the dearth of prior research addressing the objectives of this study, the conceptual development and empirical support that justify presenting formal hypotheses are lacking (Somers 2001, 187). Rather, I propose three research questions indicating characteristics of compliance with the WAPOR rules by the published survey research examined here (Neuendorf 2002). Findings regarding these questions may lead to formal hypotheses that can be investigated in future research.
Research question 1: As a class, do the qualifying articles gleaned from the 75 tourism journals in 2013 show compliance with the WAPOR rules in most of the possible cases? The 45 articles were assessed against the rules in the WAPOR Code producing 450 possible cases of compliance. Specifically, do most of the total cases comply with the WAPOR rules? The answer indicates the extent of overall compliance with the WAPOR standards by the entire sample of articles. Research question 2: What proportion of these articles comply with a majority of the WAPOR rules? If most do, then most noncompliance is centered on a minority of the articles and lack of transparency is not a general issue. But if most do not comply with most of the rules, then noncompliance is widespread across articles. Research question 3: What proportion of the rules are widely ignored? This is indicated by the percentage compliance with each rule across all articles. If only a minority of the rules are widely ignored, then noncompliance is less of an issue than if a majority of the rules are widely ignored.
Methodology
The author directed two undergraduate students in gathering and analyzing data on articles in tourism journals employing probability sample surveys in order to answer the research questions. Following is the process employed.
Secure a master list of tourism academic journals publishing articles in 2013.
We began with a list of 109 journals employing the double-blind review process and devoted to publishing research in the English language focused on one or more of the following subjects: tourism, hospitality, sports, recreation, leisure, events, meetings, and conventions. I compiled this list over two decades by referring to other lists and collecting announcements of new journals posted on the web (the master journal list is available from the author).
2. Identify the volume number for each journal associated with most of the issues published in 2013 (called the “2013 volume”).
The team searched websites for each of the journals for the volume that was most closely identified with calendar year 2013. This was defined as the volume with a majority of the issues published during the year or the volume with half of the issues released in 2012 and half in 2013. We found that nine of the journals on the master list did not publish any issues in 2013, leaving 100 journals available for this study.
3. Select a random sample of the issues stratified by number of issues published in each 2013 volume.
From journal websites, we determined how many issues were published in the 2013 volumes for each of the 100 journals. The range was from one to eight issues. The mean number of issues published was 3.6, the median was 4, and the mode was 4. More than three-quarters of the journals published 4 or fewer issues during the year.
Fourteen journals published one issue for the year so that issue was selected for the sample. For the remaining 86 journals with two to eight issues for 2013, we generated random numbers through Microsoft Excel’s RAND function to identify one issue for analysis from each journal. This produced 100 specific journal issues to be reviewed for this study.
4. Obtain copies of each article in the selected issues.
The team sought copies of the 100 selected issues through our university library in print or electronic format. Of the 100 issues pursued, 75 could be secured during the period available for this phase of the study. The 25 missing issues could not be secured through the library’s direct access or interlibrary loan during that period despite repeated efforts.
5. The two students (“coders”) read every research article in the selected issues and identified those employing probability sample surveys.
A “research article” was defined as any article appearing in a selected issue that was not a book review, conference report, editor’s commentary, case study, announcement or advertisement. Since previous ethics studies have stressed the importance of coder training (Krippendorff 2004; Neuendorf 2002), I conducted training sessions for the two student coders. These training sessions familiarized them with the purpose of this research and the content to be coded, as well as coding procedures and their applications. We employed manifest coding, where only specific words or short phrases were coded. Words used in this application included survey, sample, random, and generalizable. This is a straightforward coding process and highly reliable (Bernard, Wutich, and Ryan 2017, 249). I did not compute inter-coder reliability because lack of coding experience and expertise of the student coders violated a requirement for computing a valid measure of observed agreement (Krippendorff 2004, 217). Moreover, because of the limited time available from the coders for their task, they compared notes with each other and sought my advice during the coding process. This violated another of Krippendorff’s requirements (2004, 217). On the other hand, it was a relatively straightforward process for me to check their findings and make corrections.
Student training comprised two phases. Phase one employed practice coding of five selected articles as to whether each used a probability sample survey or not. The coders filled out a coding form for each article identifying eligible papers that employed a survey of a sample of a larger population selected in a way that the results from the sample are generalizable to their larger populations. Then I met with the two coders to go over their coding to identify errors and ensure understanding of the coding procedures.
After completing this training session, both of the coders read each of the 468 research articles published in the 75 selected issues to determine which articles employed probability sample surveys according to the coding form. The coders were instructed to be liberal in their assessments, that is, to accept an article if compliance with the survey criteria was unclear. I resolved disagreements among the coders by reading the articles in question and making the final judgments. Forty-five articles employing probability sample surveys in 2013 were identified from 31 different journals, as listed in Table 1. These 45 articles are the units of analysis, “the element on which data are analyzed and for which findings are reported” (Neuendorf 2002, 13). Expanding this to the population of 75 tourism journals suggests a total of 421 articles utilizing probability sample surveys were published in the 75 tourism journals for 2013.
6. Translate the WAPOR Code of Ethics into a set of explicit binary rules of compliance or noncompliance to be applied to the qualifying articles.
Tourism Journal Articles Employing Probability Sample Surveys in the Sample of Tourism Journals for 2013.
Source: Author.
Table 2 shows the specific coding criteria defined for each of the 11 rules from the WAPOR Code for application to the selected journal articles. WAPOR Rule g in Table 2 was not applied because it was too ambiguous to define a specific criterion and appears to be covered by other rules. Rule j was not applied because of the difficulty of determining if results reported were “based on parts of the sample, rather than the whole sample” in a journal article. This rule would be applicable to a complete survey report rather than to a journal article summarizing a report’s contents. Moreover, unless an author explicitly states that results analyzed are based only on part of the entire sample, readers cannot know this. Hence, compliance with this standard cannot be proved or disproved.
WAPOR “Rules of Practice Regarding Reports and Study Results” and Coding Criteria Applied to the Sample of Tourism Journal Articles Employing Probability Sample Survey Methods.
Sources: World Association for Public Opinion Research 2011, and author.
For most of the remaining WAPOR rules in Table 2, the compliance rule can be clearly stated and, if not met by the journal article, noncompliance is recorded for that rule. In other words, for most rules, there are only two possible, mutually exclusive states: compliance or noncompliance. However, in three cases (Rules a.2, e.1, and f), each rule is only applicable under certain conditions. If the condition for such a rule is met in an article, then compliance or noncompliance can be determined. Otherwise, such assessment cannot be made. So three states are defined for these three rules: compliance, noncompliance (i.e., information is provided indicating the rule applies but the rule’s requirement is not met), and not applicable (the required condition for application is not met and compliance or noncompliance cannot be determined). Finally, WAPOR Rule d was broken down into three compliance codes, and rule e was expressed by two criteria for coding the articles. Consequently, 13 codingcriteria for compliance were defined for the WAPOR Code of Ethics “Rules of Practice Regarding Reports and Survey Results” for this study.
6. Each of three coders coded each eligible article for compliance with the WAPOR Code of Ethics.
This study applies the ethics auditing function recommended by Fennell and Malloy (2007, 84): “assessment of performance against [a] goal or other stated criteria.” Through content analysis, often used in assessing tourism ethics (Fennell 2006), we applied the “Rules of Practice Regarding Reports and Survey Results” from the WAPOR Code of Ethics discussed above to identify compliance and noncompliance of the specific articles employing probability sample survey methods. Manifest coding was again applied at this stage. Each coder sought words or phrases and related numerical values as evidence of reporting the information required by each of the WAPOR rules.
A second training session was held by the author with the two student coders to go over the procedures for identifying how each article conformed to the WAPOR code of ethics. First, coders were required to complete a coding form for each of five eligible articles on conformance with 13 criteria operationalized from the WAPOR code. I then met with the coders to go over the results to resolve coding disagreements and answer questions about the process. Then the two students and I read each of the remaining 40 articles and recorded compliance with the WAPOR rules. Finally, I reviewed the three sets of compliance assessments and resolved disagreements among us coders.
Results
The 13 coding criteria in Table 2 were applied and three were found to be not applicable to any of the articles:
(a.2) Noncompliance = article states that the survey was conducted for or funded by an unidentified sponsor or client; otherwise, not applicable;
(e.1) Noncompliance = article states that the survey conducted differed from the survey plan without any explanation of the differences; otherwise, not applicable;
(f) Noncompliance = article states that responses were weighted prior to analysis but weighting procedures are not explained; otherwise, not applicable.
This left 10 WAPOR criteria that applied to these articles. Research question 1 addresses whether the complete collection of these articles complied with most of the WAPOR rules. The 45 articles produce 450 cases of compliance or noncompliance overall. The 45 journal articles overall complied with 53% of the WAPOR criteria across these 450 cases. This indicates that, overall, noncompliance with the WAPOR rules is substantial, but not overwhelming among the articles investigated.
Research question 2 considers the level of compliance of individual articles with the WAPOR rules: what proportion of the articles comply with a majority of the 10 criteria? Table 3 summarizes the degree of this compliance. It shows that 44% of the articles complied with more than 50% of the coding criteria. So only a minority of the articles comply with a majority of the WAPOR standards. Fifty-six percent of the articles complied with one-half or fewer of the criteria.
Summary of Sample Articles’ Compliance with WAPOR Ethical Rules.
Source: Author.
Research question 3 deals with the distribution of noncompliance among the rules: is it widespread or confined to relatively few of them? Table 4 shows the results of assessing the 45 articles’ compliance with each of the 10 coding criteria from the WAPOR Code of Ethics in ascending order of compliance. Only three of the criteria (the last three in Table 4) are observed by 90% or more of the articles. On the other hand, three of the criteria are complied with in fewer than two in five articles.
Compliance of the Sample of Tourism Journal Articles Employing Probability Sample Survey Methods with the WAPOR Rules of Practice.
Source: Author.
Conclusions
This is a study in ethical reflexivity—observation of and reflections on the ethical nature of current tourism research employing a specific method. Such investigations are essential if knowledge of our field is to grow and contribute to practice in constructive ways. The present study addresses two issues that have received little attention in tourism academic journals: deriving a set of ethical standards incorporating the value of research transparency in published research using a specific methodology, and applying and evaluating compliance with them in published tourism research.
Research articles from a relatively large number of tourism-related journals in 2013 were selected and examined to identify those employing probability sample research methods. Forty-five qualifying articles were found. Then a professional code of ethics promulgated by the principal international association of sample survey researchers—the World Association for Public Opinion Research (WAPOR)—was operationalized in 13 coding criteria for assessing methodological transparency. Ten of these criteria were found to apply to the 45 articles. Coders assessed compliance of each of the articles with the 10 criteria. Overall, the assessment found compliance in just over one-half of the cases. Moreover, only a minority of the articles complied with most of the criteria. Finally, only 3 of the 10 criteria were observed by 90% or more of the articles.
We conclude from this study that articles employing probability sample surveys in tourism journals can be assessed as to their compliance with a widely recognized international code of ethics expressing the value of transparency of method for publishing such research. The procedures described above can be applied to other journals at other times to assess their compliance with the ethic of research transparency, a foundation of valid social science research. On the second issue in research reflexivity addressed here, it is clear that the WAPOR code is currently not effectively promoting such ethical compliancein tourism journals. Consequently, readers are not protected from misrepresentation of results, journal referees are hampered from fairly assessing submitted manuscripts, and users have reduced confidence in the research (Fennell 2013). The question naturally arises as to why. It appears likely to this researcher that authors of studies based on probability sample surveys and the editors of the journals where such articles are published are not aware of the World Association for Public Opinion Research Code of Ethics, or the similar codes promulgated by the American Association for Public Opinion Research (AAPOR) and the European Society for Opinion and Market Research (ESOMAR). If tourism journal publishers, editors, and contributors wish to improve transparency of research method in the articles they publish, a hallmark of science, they should adopt the WAPOR Code and observe the rules detailed above. Increased compliance with the WAPOR Code of Ethics in publishing probability sample survey articles in tourism journals in the future would encourage valid replication of results, promote extensions of findings into new areas, and allow readers to assess the potential for bias in the implied population parameters. Such benefits can raise the status of tourism journals and enhance readability of such articles as recommended by Dolnicar and Chapple (2015).
There is evidence that relatively little knowledge transfer is occurring from research published in tourism journals to tourism managers, owners, and other practitioners (Frechtling 2004). This failure is quite unacceptable in a world where businesses and public agencies continually face budget constraints. Surely tourism business managers and policy officials could benefit from many of the research articles published each year in academic journals. Improving the transparency of such research along with active communication of results to management can improve decision making for the benefit of tourism hosts and visitors.
Recommendations
Academic journals employing double-blind peer review processes are the lifeblood of tourism knowledge growth. They stimulate new knowledge, extend existing knowledge into new areas of study, facilitate the organization and management of knowledge, and identify distinct areas where knowledge is insufficient. But they can succeed in their missions only if they observe ethical standards of scientific research and maintain confidence among their audiences that what they publish is sound and worthy of application. Readers of research employing probability sample surveys applied to tourism topics cannot now have that confidence.
Fortunately, the remedy for the current lack of ethics of transparency in articles employing probability sample surveys is straightforward. Editors of tourism-related journals can require that articles submitted to them indicate if the methodology comprises drawing a sample from a larger population in order to statistically represent that population. If so, the authors of such articles can be required to report in the article each of 14 characteristics of their probability sample surveys shown in Table 5. This list includes all of the WAPOR rules as well as three additional standards that ensure complete transparency of method for replication purposes.
Characteristics of the Probability Sample Survey Employed in This Study.
Source: Frechtling (2004).
If published as part of the text, this information vastly improves the transparency of the sample survey research process. In addition to the benefits for readers and researchers, such standard information will assist referees in assessing the quality of the articles they review and in identifying specific shortcomings. There is precedent for requiring researchers to submit details of their methodologies. Tourism Management requires authors of manuscripts using quantitative methods to submit information about their methods in five areas, for qualitative research, five areas, and for mixed methods, similar information. The editors specifically state, “Every attempt must be made to ensure transparency as to the nature of the data and its collection for the benefit of referees and potential readers” (Tourism Management 2016, 6). The recommendations here extend the research transparency sought by this respected journal.
Future Research
If journal editors require authors to submit WAPOR Code information as suggested here, future studies can then explore developing criteria of excellence for probability sample surveys. That is, editors can specify thresholds for each of the 14 characteristics in Table 5, such as minimum effective sample size and minimum response rate, that must be met for the manuscript to be sent to editorial reviewers.
Should tourism journal editors adopt these recommendations, repeating this study in five years or so can suggest what progress has been made toward increasing ethics in this critical research area. Were the current study applied to audit journals in other fields (e.g., marketing, education, economics) for WAPOR Code compliance, we could assess how far ahead or behind tourism journals are relative to journals in other fields in observing the ethics of presentation of probability sample survey results.
Finally, the audit procedures employed in this study can be extended to other methodologies. For example, AAPOR has published a list of disclosure items required to ensure transparency in publishing qualitative research and another for publishing papers employing content analysis (American Association for Public Opinion Research 2016, section III.B and C). Information required for expanding the transparency of other research methods employed in tourism journal articles (e.g., structural equation modeling, economic impact estimation, and forecasting models) can also be developed and applied. Such expanded transparency of research method should produce significant, positive effects for journal editors, researchers, and the wider audiences who can benefit from their research.
Footnotes
Acknowledgements
The author gratefully acknowledges the assistance of School of Business students Mr Jayden Liu and Mr Alexander Lee, whose assistance was critical to this study’s conduct and conclusion.
Declaration of Conflicting Interests
The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The research described in this article was supported by grants from the International Institute of Tourism Studies and the Office of the Dean of the School of Business, George Washington University.