Abstract
Scholars in scientific disciplines face unique challenges in the creation of visualizations, especially in publications that require insights derived from analyses to be visually displayed. The literature on visualizations describes different techniques and best practices for the creation of graphs. However, these techniques have not been used to evaluate the impact of visualizations in academic publications. In the field of ecology, as in other scientific fields, graphs are an essential part of journal articles. Little is known about the connections between the kind of data presented and domain in which the researchers conducted their study that together produces the visual graphics. This study focused on articles published in the Journal of Ecology between 1996 and 2016 to explore possible connections between data type, domain, and visualization type. Specifically, this study asked three questions: How many of the graphics published between 1996 and 2016 follow a particular set of recommendations for best practices? What can Pearson correlations reveal about the relationships between type of data, domain of study, and visual displays? Can the findings be examined through an inter-reliability test lens? Out of the 20,080 visualizations assessed, 54% included unnecessary graphical elements in the early part of the study (1996–2010). The most common type of data was univariate (35%) and it was often displayed using line graphs. Twenty-one percent of the articles in the period studied could be categorized under the domain type “single species.” Pearson correlation analysis showed that data type and domain type was positively correlated (r = 0.08; p ≤ 0.05). Cohen’s kappa for the reliability test was 0.86, suggesting good agreement between the two categories. This study provides evidence that data type and domain types are equally important in determining the type of visualizations found in scientific journals.
Introduction
Visual displays of data are used to facilitate an understanding of complex numerical representations of scientific concepts or empirical study results in almost all scientific fields. 1 Creating those visual displays demands significant time and effort, 2 and also creates a bottleneck at the review stage in fields that require systematic judgment of the quality of the visual displays. 3 Nevertheless, modern expectations for publishing in a scientific journal require authors to communicate their findings via both text and visualizations. 4 Consequently, data visualizations are used more frequently now than in the past.5–7 The field of ecology was chosen as the focus of this paper because research in this field emphasizes diverse and complex relations among different populations and species. As the literature in the field shows, the complexity of the data strongly influences the data analytics, together with the research questions about the domain of investigation directly impacts the choice and production of graphic displays to communicate the findings. General discussion on data visualization in the field has focused on different methodologies for data collection and analytical tools, together with the research questions and specific information about how the data are communicated. Greenacre 8 and Ellison 9 argued that the presentation of visual data could be improved in academic publications, even though manuscript reviewers are often more focused on the text than the graphics. Ellison 9 further outlined best practices for the creation of data visualizations based on work by Tukey, 10 Tufte, 11 and Cleveland. 12 However, the relationships between visualizations in journal publications and the type of data and domain they represent have never been examined. To date, relatively few studies have considered how visualizations are used in different subfields or disciplinary domains, and the extent to which the creators of the data displays follow best practices for data visualization.
This study addressed the gaps in knowledge among data type, domain type, and visualization by assessing visualizations produced by researchers and published in a leading ecology journal, the Journal of Ecology. This study asked the following questions: How many graphics in the Journal of Ecology between 1996 and 2016 followed the best-practice recommendations of Ellison 9 and Tufte 11 for displaying visualizations in the field of ecology? Do Pearson correlation relationships test between the type of data, type of domain, and the visual display used to present findings? Can our findings be examined through an inter-reliability test lens? More specifically, through Cohen kappa (κ) scores.
Visualizations in academic journals
The commonly used phrase, “A picture is worth a thousand words,” is testimony to the power of visualization. The term visualization traditionally refers to two actions; first, it may describe a human cognitive activity. Second, it may refer to the action of conveying data in a graphical, visual form. 13 Isenberg et al. 14 expanded upon this understanding of visualization by recommending a different approach to visualization which emphasizes “collaborative visualization.” The authors noted that visualization as a collaborative activity allows the scientific community to go beyond a single methodology or software development. An early definition of the term collaborative visualization proposed by Raje et al. 15 emphasized the contributions of different experts toward a shared goal of understanding the visual object, phenomenon, or data under investigation. This study adopts the definition put forth by Raje et al. 15 to examine work in ecology conducted by different ecological experts. Here, we surveyed visualizations published in a leading ecological journal to better understand the factors that influence visualizations, such as data type and domain of study.
According to Nielsen et al., 16 the source of success of scientific visualizations lies in the premise behind them; computer-generated pictures are used to gain information and understanding from data (geometry) and their relationships (topology). Many researchers have stated that visualizations make quantitative information easy to understand.17–19 Visualizations assist in conveying quantitative or scientific concepts to the reader, particularly when the concept is explicitly expressed in the display.20,21 Tufte11,22 and Bertin 23 published highly influential books concerning the visualization of quantitative information. Bertin 23 developed a guide to the selection of a graphic form and its design. Tufte11,22 outlined simple principles for graphical excellence and integrity. A different approach for the effective conveyance of visual data was proposed by Cleveland,14,44 who argued that visual models invariably depend on the data used and proposed a classification scheme of visual displays based on the type of data used.
Recent developments in technology have made it possible to generate graphs and charts quickly and easily. 24 However, visualizations may contain errors or misrepresent the data, despite the technological progress.25,26 The growing popularity of different guidelines for effective scientific visualizations has led to a recognition of the importance of examining the structure and grammar of visualizations in scientific journals.27–30 However, no previous study has empirically examined visualization displays in the field of ecology.
Visualization as a scholarly activity has been examined outside the field of ecology, researchers value the contribution of visualizations to the domain in which the work is produced. In the field of psychology, Zacks et al. 31 found that the mean number of graphics in academic journals nearly doubled between 1984 and 1994. Reporting on visual representations in nutrition journals, Magnet 32 found that the number of visual displays had increased since the 1960s, although no single type of display appeared more frequently across all journals. In the field of information science, Friedman 33 examined the relationship between the type of research method used and the type of visualization presented in the findings, based on contributions to conference proceedings. He found no direct relationship between the type of research methodology and type of visual display; instead, many researchers customized their research methods and their visualization patterns to fit their specific needs. To assess the relationships between articles and visual displays, Friedman used a visual content analysis in which each article was assessed manually. Munzner 34 reported on the process of writing information visualization articles for IEEE-Vis conferences. She described the different stages involved in this process, including decisions required by the researcher with respect to the type of paper, validation approach, technique, study design, system, model, and evaluation of the paper. She also reported on pitfalls researchers encounter encoding visualizations in preparing manuscripts, especially visualizations in which straightforward design approach is often not sufficient. A different approach was taken by Howe et al., 28 who built a web-based search engine that allowed users to search for visualizations in scholarly work based on type. This platform extracted visual information from the scientific literature. However, none of the above studies reported on an empirical examination of the relationship between the type of data collected in the study and the type of visualization in a specific domain.
Many researchers in the field of ecology have cited Ellison 9 as a primary reference for their work on data visualization.35–37 Marcum 38 discussed the challenges to data visualization in ecology from the perspective of librarianship and human society. In this paper, he introduced the concept of visual ecology, in which the emphasis is on observation and description, and a more intensive focus on people and information-related behavior is utilized. In order to convey these concepts in the world of visual ecology, he employed the design principles of Tufte 11 to demonstrate solutions meeting the varying demands of visual observation and description. Ellison 9 took a more traditional approach by examining different statistical methods and analyses and their applications to type of visualization, with reference to the work of Tukey, 10 Tufte, 11 and Cleveland. 12 He examined Cleveland’s discussion regarding data analytics in the field of ecology and compared it to the principles laid out by Tufte. Ellison reported that graphics serve the following two functions: to explore patterns in the data, and to clearly communicate the output of the analysis. In this paper, Cleveland’s approach was used to examine the visualization type in the Journal of Ecology because of the impact it had on researchers in the field of ecology, and the pragmatic nature of categorical classification between type of data and type of visualization.
Researchers have also reported on the need to add scientific domain to the investigation of the data discovery of new knowledge through collaboration by researchers examining common theories.39,40 The investigation of domain as a visual effect on knowledge has been addressed by Cohen’s CiteSpace.41–43 This tool aims to analyze bibliometric data that produce different visualization methodologies and visualization types that reveal how knowledge domains evolve. However, these studies did not examine the type of visualization that the authors used in their publication.
Related work
A review of academic publications in ecology (journals, conference proceedings, dissertations, and monographs) revealed that the use of data and visualizations by researchers varies more widely in journal articles than in any other type of publication. The literature in ecology indicates that data type and domain of study influence the choice of output by researchers. 45
Data type
Ecological and biological data are often characterized by unusual distributions that do not resemble an idealized, bell-shaped and symmetric normal distribution. 8 Most textbooks on statistics divide data into the following three common formats: univariate, bivariate, and trivariate. 46 Cleveland 12 added two more classifications—hypervariate and multiway. Hypervariate data contain four variables, and multiway data contain five or more variables. However, he added, “measurement of four or more quantitative variables are hypervariable data” (p. 303). Zuur et al. 46 and Ellison 9 found that using the data classification scheme outlined by Cleveland helped researchers formulate their visual displays. This study used the data classification scheme described in Cleveland12,47 to categorize the visual displays under consideration.
Visualization type
Data in a research article risk being misinterpreted or ignored if they are not properly designed and presented. Graphs and figures can save readers time and energy and facilitate a better understanding of the data and analysis in the article.21,27,33,48 Shah and Hoeffner 47 raised the following question: why are some graphs relatively easy for viewers to comprehend for a particular task, whereas other graphs are more difficult? They outlined three major component processes of visual images along with several factors that influence visual conceptual relations presumed from a visual display. The first factor is graph comprehension, which is encoded by visual features; the viewer needs to identify the visual features of the layout (such as a line graph that consists of two axes, x and y). The second factor is the visual component process that compares the visual features of the specific visual display with other, referent visual displays. The third component of graphical understanding is that the viewer must determine the reference of the concept being quantified and associate that referent with the coded function. 20 Each of these factors influences how a visualization is perceived and interpreted. However, neither Shan and Hoeffner 47 nor Bertin 23 address how data type or the domain of study affect viewer comprehension. As outlined above, Cleveland 12 classified visualizations based on five types of data. Univariate data are often shown as quintile plots also known as Q-Q plots, box plots, scatterplots, line graphs, or random-dot stereograms (RDS). Bivariate data may be shown as line graphs, Q-Q plots, box plots, scatterplots, line graphs, scatterplots, scatterplots fitted with a with smoothing curve, density plots with a smoothing parameter, time-series plots, or multi-plot lattice graphs. Trivariate data are often shown as line graphs, conditioning plots, level plots, contour plots, 3D wireframe plots, level plots of surfaces with superposed color, and superposed level region color graphs. Hypervariate data may be shown as scatterplot matrices, co-plots, cropping charts, conditioning plots, level plots, contour plots, or classification plots. Cleveland associated multiway data, which is characterized by having more than five variables, with dot plots, superposition plots, and exploratory dot plots. Table 1 shows Cleveland’s associations between data and visual representations. 12
Classification of visualizations based on data type, from Cleveland. 12
In this study, we classified visual displays based on intervening variables (data type and domain type) in order to determine whether a causal link exists between the type of study data and type of visualization used to present those data.
Ecosystem domains
Ecologists often divide their field based on different ecosystems with different benchmarks. 49 In ecology, the term domain refers to the spatial and temporal patterns of the distribution and abundance of organisms, including causes and consequences. 50 The classification of domains in ecology has been considered by two leading societies in the field, the British Ecological Society 51 and the Ecological Society of America. 52 These societies conducted interview surveys to examine the factors that affect researchers in their choice of the domain in their publications. They reported that the establishment of domains in the field of ecology strengthened the experience and proficiency of researchers. Ecologists often describe a general theory of ecology that consists of a description of the domain of ecology and a set of fundamental principles. 53 A different understanding of domains in ecology was expressed by Monath, 54 who reported that domains in the field were in transition. Carmel et al. 55 described trends over the last 30 years regarding ecological domains. The authors identified five broad domains in ecological literature; single species, species interactions, community, ecosystem, and other. The domain of single species included topics such as demography, physiology, distribution, behavior, evolution, and genetics. The domain of species interactions included studies of grazing, predation, mutualism, parasitism, and competition. The domain of community included studies of biodiversity and community structure. The fourth domain, ecosystem, included studies of food webs, climate change, vegetation dynamics, biomass and productivity, and biogeochemistry. The final domain, “other” included studies focused on scale and statistics. Ecological researchers have reported that the areas of basic research interest and the methodologies used to conduct research have not evolved substantially over the last 30 years. 51 This study assessed ecological journal articles and classified each according to one of the five domains outlined in Carmel et al. 55 The findings of Carmel et al. 55 are shown in Table 2.
Frequencies of the five domains identified in ecological studies.
Source: Data from Carmel et al. 55
Best practices
Several researchers have discussed best practices for the visual display of data.9,19,22,51 Kelleher and Wagener 56 outlined 10 categories of best practices for effective data visualization in scientific publications. The guidelines they outlined address common issues in presenting data visually in academic publications. A different approach was proposed by Wongsuphasawat et al., 57 who created a visualization tool that automatically visualizes data based on specific data and domain. However, neither of these studies are referenced in the ecological literature. The discussion of statistical graphs in ecology-focused papers most often makes reference to Ellison, 9 who outlined four guidelines for graphics drawn from Tufte, 11 Tukey, 10 and Cleveland. 12 These guidelines are as follows:
Underlying patterns of interest should be illumined, while not compromising the integrity of the data.
The data structure should be maintained, so that readers can reconstruct the data from the figure.
Figures should have a high data: ink ratio and no chartjunk. The latter term was defined by Tufte11,22 as any visual graphical element that is not necessary for comprehension of the information in the graph. Chartjunk distracts the viewer from the data analysis.9,22 The term focused on transparency the graph must display.
The visual figures should not be distorted, exaggerated and/or censored. 9
This study investigated whether the principles described by Ellison 9 and Tufte 11 are commonly followed in article figures in the Journal of Ecology. The rationale for using the best practices of Ellison and Tufte as a guideline came from the popularity among ecology researchers, who often have cited their work.
Methodology
This study examined articles published in the Journal of Ecology (1996–2016), with the aim of empirically determining how researchers in the field of ecology utilize visual displays. Our methodology consisted of visual content analysis. We based our methods on previous studies that outlined visual content analysis as a visual collection process and procedure58,59 together with different case studies.34,60,61 Content analysis commonly involves the management of a large data collection in which researchers divide a set into categories, variables, and code to determine whether those categories, variables, or code fit the data. The visual content analysis of this study involved four rounds of analysis: coding the categories, analysis of the entire dataset, analysis of a random subset of the data, and inter-rater reliability analysis.
In the first step, we recorded every paper appearing in the journal from 1996 to 2016, together with the type of visualization used, the type of data, and domain referred to by the authors. We then sorted this data collection based on three main categories: type of visualization, type of data, and type of domain. In the second step, we analyzed the data collected by categorizing it based on the entire dataset of articles. In the third step, we conducted a sample to assess the relationships between data type and domain type with visualization type. In this stage, we used the Pearson correlation coefficient. In the last stage, we conducted an inter-rater reliability test using Cohen’s kappa coefficient (κ) test to validate our results.
We present two examples to illustrate the second step of our analysis. In the first example, Royer et al. 62 used a line graph to compare ground radiation levels between sites. In this study, the authors described an experimental research methodology that generated trivariate data. They examine the growth of animal and plant populations relating to ground reflection and regional ecological dimensions. Figure 1 shows the visualized data. Our study classified this research in the domain of ecosystem under the sub-category topic. In the second example, Kozlov et al. 63 used a bar graph to display data from which two variables were selected, climate zones and consumed leaf area. The authors classified their investigation as pertaining to ecosystems (topic, vegetation dynamics). We classified the domain as species interactions (topic, mutualism) because the article described plant-herbivore interactions with respect to climate zone. Figure 2 shows the visual display of data from Kozlov et al. 63
Visual display of ground radiation and canopy cover.
Bar graph summarizing different conditions for leaf growth.
First, we analyzed all articles from 1996 to 2016 using descriptive statistics. We noted the type of data used in the study, the domain of study, and the most frequently used type of visualization, together with the best practices used in those visualizations. In the second phase of analysis, we conducted a sample to assess the relationships between visualization type, best practices, data type, and domain type. We tested our hypotheses using Pearson’s correlation coefficient (also known as Pearson’s r). We selected the Journal of Ecology for this study because of its easy accessibility and inclusion of visual displays. The journal was founded in 1913 and has a long history of publishing high-quality ecological research, in which the type of data and type of domain became a common premise in the titles and key words describing manuscripts. A general review of the Journal of Ecology publication revealed that visualization graphs appeared in the journal ever since early 1960s, but its significant frequency only occurs ever since the late 1990s.
In the text that follows, it is important to remind readers that the study reviewed the totality of articles, and thus is not a sample. Therefore, observations of article frequency are significant to the set of articles under study, but not beyond. On the other hand, we are also working with a “sample”; we also examined statistical significance between the two categories, data type and domain type. Using Cohen’s kappa coefficient (κ score for inter-rater reliability to test various hypotheses, we report with 95% confidence the results are not likely to occur by chance.
Results
In the first step of the analysis, we assessed the entire set of articles published between 1996 and 2016. The final dataset included a total of 20,080 visual displays. An average of 66.7 visual displays appeared in each journal issue. Eighty-five percent of the articles used at least one visual display. The most productive year, in terms of visual displays, was 2014; during this year 675 visual displays were published. The year with the fewest published visual displays (154) was 1996. Figure 3 shows the number of visual displays published in each issue per year.
The number of visual displays published in each issue per year.
Next, we reviewed the entire dataset of articles to assess whether these visualizations followed the best practices outlined in Ellison 9 and Tufte.9,22,64 Out of the 20,080 total visualizations assessed, we identified a total of 1,365 cases of unnecessary graphical elements (average, 11.5 per issue). We expanded our analysis to evaluate the overall performance by counting the number of unnecessary images found in each issue of the journal and adding them to obtain the sum for that particular year. A Hidden Markov model 65 showed that the greatest frequencies of visualizations containing unnecessary graphics and other chartjunk occurred between 1996 and 2010. Unnecessary graphics were present in a total of 54% of the visual displays. The average number of unnecessary visual elements found per issue was 15.4 between 1996 and 2010, which was substantially higher than the average of 3.8 unnecessary elements observed between 2010 and 2016. Figure 4 shows the number of unnecessary elements included in graphical displays in comparison to the overall number of graphs over time (1996–2016).
The total number of unnecessary graphics found in the Journal of Ecology between 1996 and 2016.
We also calculated the type of visualization that was most often used during the period of study. The dominant type of visualization was the line chart, which accounted for 24% of all visualizations. Twenty-one percent of the visualizations were bar graphs, and 15% were histograms. Visualizations that did not fit one of the classifications outlined by Cleveland 12 were categorized as unidentified; these displays included scanned images and hand-drawn graphs. Figure 5 shows the number of different types of visualizations published in the Journal of Ecology over the study period.
The most common visualization types between 1996 and 2006.
Due to the lack of literature on the subject of the relationship between domain type and data type, we examined the distribution of the domain of the study and the type of data presented in the entire dataset of articles, side by side. The most common type of data used was univariate (35%). Twenty-seven percent of the data were bivariate, and 16% were trivariate. We were unable to identify the data type for 5% of the articles examined due to lack of clarity in the data description. Under the study of domain, single species was the most common domain, with 31% of all the articles that contained visual displays classified under this category. The domain of species interactions accounted for 19% of studies, and community accounted for 17%. Figure 6 shows the distribution of studies published in the Journal of Ecology according to domain and type of data over the study period. We were unable to identify the domain for 4% of the articles (24/620) due to a lack of clarity in article domain description. This result was consistent with Camel et al. 55 findings.
Distribution of studies published in the Journal of Ecology between 1996 and 2016.
In the third step of our analysis, we analyzed a randomly selected sample of the data in order to address our second question: What can Pearson statistical correlation tell us about the relationships between type of data, domain of study, and visual displays? In total, our sample included 575 articles containing 251 visualizations. The random sample was determined with a 95% confidence level and a 0.11 confidence interval. Our analysis of Chi-squared tests of independence between the two categories reveal the score χ2 (1, N = 251) = 1.7, where the p value was ≤0.05. This result indicated that there is a high correlation between our two sets of data.
We used a Pearson’s correlation coefficient analysis to test the correlation between the two categories. Results indicated that there is a strong positive linear relationship between data type and domain type (r = 0.81, df = 43, t = 2.474, p ≤ 0.05). The results confirm a correlation coefficient of 0.81, confirming that there is a strong positive linear relationship between data type and domain type. The p-value is also below 0.05 so the null hypothesis can be rejected in favor of the alternative. Figure 7 summarizes the results of the Pearson correlation analysis.
Summary of Pearson correlation analysis based on ggplot2.
In the last step in our analysis, we used Cohen’s kappa (κ) to measure the degree of inter-rater agreement between our two main categories, data type and domain type, following the recommendations outlined in Bakeman et al.66,67 We measured data type and domain type and evaluated the degree of agreement based on Cohen’s kappa. Analyses were conducted in R using the vcd package. 68 Our results showed that κ = 0.89, indicating strong agreement between the categories.
The results showed a linear relationship between the domain type and data type. We used Pearson correlation analysis to determine the statistical significance of this relationship. For this analysis, r = 0.81 and p≤ 0.05 (n = 251). We also measured the degree of inter-rater reliable dependability between the two main categories, data type and domain type, based on Cohen’s kappa. We found strong evidence that the linear relationship between data type and domain type is reliable.
The objective of this study was to examine correlational connections between the type of visualization presented in a journal article, the type of data collected, and the domain of the research. We found that domain type and data type have direct relationships with visualization type. The findings from this study also contribute to a larger discussion of best practices for data visualization in academic publications by showing changes in the visual representations of data by ecology researchers since 1996. Future studies should examine the effect of visual grammar graphs on the ability of scientific researchers to communicate their findings; more specifically, whether following visual grammar rules provides a more detailed outline for creating graphs and charts.
Conclusion
Scientific graphs and charts, also known as visualizations, are important to the study of ecology and scientific publications. However, according to Greenacre, 8 publications authored by ecological researchers commonly feature inappropriate visualizations and/or statistical analyses. The objective of this study was to empirically examine the graphs and charts published in a leading ecology journal, the Journal of Ecology, with respect to the type of data presented, the ecological domain, and use of best practices. This study had four main findings. The first was that visualizations have played a role in the dissemination of knowledge through the Journal of Ecology. We also found that line graphs have remained a popular form of data display in the journal throughout the years (1996–2016). An assessment of whether the visual displays in the journal followed best practices revealed that more recently published articles followed the rules for good graphics outlined in Ellison 9 more closely than did earlier publications. Last, Pearson correlation analysis and a comparison revealed that the ecological domain and data type have a strong linear association with reference to visual display. Finally, an inter-rater reliability test based on Cohen’s kappa indicated that our results were consistent.
This study provides empirical evidence for the role of visualization in scientific discovery in the field of ecology. It highlights that domain type and data type are equally important in determining the type of visualization in a scientific journal. Although the subject of scientific visualization continues to evolve, visual grammar has become a platform for analyzing data.34,45 Future studies should examine and measure the visual grammar proposed by Wilkinson70 as a criterion for measuring the visualizations found in the ecology literature. Exploring this type of visual rubric will provide researchers with guidance on how to measure the effectiveness of visualizations found in the scientific literature. Future studies may also address the various visual grammar models that are now being used to generate visualizations for specific journals. Finally, future studies should determine whether visual grammar can provide insight into the choice of visualization, and guidance on how to measure the memorability of visualizations found in scientific journals.
