What difference does quantity make? On the epistemology of Big Data in biology

Introduction

Big Data has become a central aspect of contemporary science and policy, due to a variety of reasons that include both techno-scientific factors and the political and economic roles played by this terminology. The idea that Big Data is ushering in a whole new way of thinking, particularly within the sciences, is rampant – as exemplified by the emergence of dedicated funding, policies and publication venues (such as this journal). This is at once fascinating and perplexing to scholars interested in the history, philosophy and social studies of science. On the one hand, there seems to be something interesting and novel happening as a consequence of Big Data techniques and communication strategies, which is, however, hard to capture with traditional notions, such as ‘induction’ and ‘data-driven’ science (partly because, as philosophers of science have long shown, there is no such thing as direct inference from data, and data interpretation typically involves the use of modelling techniques and various other kinds of conceptual and material scaffolding). ¹ On the other hand, many sciences have a long history of dealing with large quantities of data, whose size and scale vastly outstrip available strategies and technologies for data collection, dissemination and analysis (Gitelman, 2013). This is particularly evident in the life sciences, where data-gathering practices in subfields, such as natural history and taxonomy, have been at the heart of inquiry since the early modern era, and have generated problems ever since (e.g. Johnson, 2012; Müller-Wille and Charmantier, 2012).

So what is actually new here? How does Big Data science differ from other forms of inquiry, what can and cannot be learnt from Big Data, and what difference does quantity make? In this article, I discuss some of the central characteristics typically associated with Big Data, as conveniently summarised within the recent book Big Data by Mayer-Schönberger and Cukier (2013), and I scrutinise their plausibility in the case of biological research. I then argue that the novelty of Big Data science does not lie in the sheer quantity of data involved, though this certainly makes a difference to research methods and results. Rather, the novelty of Big Data science lies in (1) the prominence and status acquired by data as scientific commodity and recognised output both within and beyond the sciences and (2) the methods, infrastructures, technologies and skills developed to handle (format, disseminate, retrieve, model and interpret) data. These developments generate the impression that data-intensive research is a whole new mode of doing science, with its own epistemology and norms. I here defend the idea that in order to understand and critically evaluate this claim, one needs to analyse the ways in which data are actually disseminated and used to generate knowledge, which I refer to as ‘data journeys’; and I consider the extent to which the current handling of Big Data fosters and validates its use as evidence towards new discoveries. ²

Accordingly, the bulk of this article reviews the development of sophisticated ways to disseminate, integrate and re-use data acquired on model organisms, such as the small plant Arabidopsis thaliana, the nematode Caenorhabditis elegans and the fruit-fly Drosophila melanogaster (including data on their ecology, metabolism, morphology and relations to other species) over the last three decades of work in experimental biology. I focus on online databases as a key example of infrastructures set up to organise and interpret such data; and on the wealth and diversity of expertise, resources and conceptual scaffolding that such databases draw upon in order to function well. This analysis of data journeys through model organism databases illuminates some of the conditions under which the evidential value of data posted online can be assessed and interpreted by researchers wishing to use those data to foster discovery. At the same time, model organism biology has been one of the best funded scientific areas over the last three decades, and the curation of data produced therein has benefited from much more attention and dedicated investments than data generated in the rest of the life sciences and biomedicine. Considering the challenges encountered in disseminating this type of data thus also highlights the potential problems involved in assembling data that have not received comparable levels of care (i.e. the vast majority of biological data).

In my conclusions, I use these findings to inform a critique of the supposed revolutionary power of Big Data science. In its stead, I propose a less sensational, but arguably more realistic, reflection on the difference that data quantity is making to contemporary biological research, which stresses both continuities with and dissimilarities from previous attempts to handle large datasets. I also suggest that the natural sciences may well be the area that is least affected by Big Data, whose emergence is much more likely to affect the political and economic realms – though not necessarily for the better.

The novelty of Big Data

I will start by considering three ideas that, according to Mayer-Schönberger and Cukier (2013) among others, constitute core innovations brought in by the advent of Big Data in all realms of human activity, including science. The first idea is what I shall label comprehensiveness. This is the claim that the accumulation of large datasets enables scientists to ground their analysis on several different aspects of the same phenomenon, documented by different people at different times. According to Mayer-Schönberger and Cukier, data can become so big as to encompass all the available data on a phenomenon of interest. As a consequence, Big Data can provide a comprehensive perspective on the characteristics of that phenomenon, without needing to focus on specific details.

The second idea is that of messiness. Big Data, it is argued, pushes researchers to embrace the complex and multifaceted nature of the real world, rather than pursuing exactitude and accuracy in measurement obtained under controlled conditions. Indeed, it is impossible to assemble Big Data in ways that are guaranteed to be accurate and homogeneous. Rather, we should resign ourselves to the fact that ‘Big Data is messy, varies in quality, and is distributed across countless servers around the world’ (Mayer-Schönberger and Cukier, 2013: 13) and welcome the advantages of this lack of exactitude: ‘With Big Data, we’ll often be satisfied with a sense of general direction rather than knowing a phenomenon down to the inch, the penny, the atom’ (Mayer-Schönberger and Cukier, 2013). ³

The idea of messiness relates closely to the third key innovation brought about by Big Data, which Mayer-Schönberger and Cukier call the ‘triumph of correlations’. Correlations, defined as the statistical relationship between two data values, are notoriously useful as heuristic devices within the sciences. Spotting the fact that when one of the data values changes the other is likely to change too is the starting point for many discoveries. However, scientists have typically mistrusted correlations as a source of reliable knowledge in and of themselves, chiefly because they may be spurious – either because they result from serendipity rather than specific mechanisms or because they are due to external factors. Big Data can override those worries. Mayer-Schönberger and Cukier (2013: 52) give the example of Amazon.com, whose astonishing expansion over the last few years is at least partly due to their clever use of statistical correlations among the myriad of data provided by their consumer base in order to spot users’ preferences and successfully suggest new items for consumption. In cases such as this, correlations do indeed provide powerful knowledge that was not available before. Hence, Big Data encourages a growing respect for correlation, which comes to be appreciated as not only a more informative and plausible form of knowledge than the more definite but also a more elusive, causal explanation. In the words of Mayer-Schönberger and Cukier (2013: 14): ‘the correlations may not tell us precisely why something is happening, but they alert us that it is happening. And in many situations this is good enough’.

These three ideas have two important corollaries, which shall constitute the main target of my analysis in this article. The first corollary is that Big Data makes reliance on small sampling, and even debates over sampling, unnecessary. This again seems to make sense prima facie: if we have all the data about a given phenomenon, what is the point of pondering which types of data might best document it? Rather, one can now skip that step and focus instead on assembling and analysing as much data as possible about the phenomenon of interest, so as to generate reliable knowledge about it: ‘Big Data gives us an especially clear view of the granular; subcategories and submarkets that samples can’t assess’ (Mayer-Schönberger and Cukier, 2013: 13). The second corollary is that Big Data is viewed, through its mere existence, as countering the risk of bias in data collection and interpretation. This is because having access to large datasets makes it more likely that bias and error will be automatically eliminated from the system, for instance via what sociologists and philosophers call ‘triangulation’: the tendency of reliable data to cluster together, so that the more data one has, the easier it becomes to cross-check them with each other and eliminate the data that look like outliers (Denzin, 2006; Wylie, 2002).

Over the next few sections, I show how an empirical study of how Big Data biology operates puts both of these corollaries into question, which in turn compromises the plausibility of the three claims that Mayer-Schönberger and Cukier make about the power of Big Data – at least when they are applied to the realm of scientific inquiry. Let me immediately state that I do not intend this analysis to deny the widespread attraction that these three ideas are generating in many spheres of contemporary society (most obviously, big government) and which is undoubtedly mirrored in the ways in which biological research has been re-organised since at least the early 2000s (which is when technologies for the high-throughput production of genomic data, such as sequencing machines, started to become widely used). Rather, I wish to shed some clarity on the gulf that separates the hyperbolic claims made about the novelty of Big Data science from the challenges, problems and achievements characterising data-handling practices in the everyday working life of biologists – and particularly the ways in which new computational and communication technologies such as online databases are being developed so as to transform these ideas into reality.

Big Data journeys in biology

For scientists to be able to analyse Big Data, those data have to be collected and assembled in ways that make it suitable to consider them as a single body of information (O’Malley and Soyer, 2012). This is a particularly difficult task in the case of biological data, given the highly fragmented and pluralist history of the field. For a start, there are myriads of epistemic communities within the life sciences, each of which uses a different combination of methods, locations, materials, background knowledge and interest to produce data. Furthermore, there are vast differences in the types of data that can be produced and the phenomena that can be targeted. And last but not least, the organisms and ecosystems on which data are being produced are both highly variable and highly unstable, given their constant exposure to both developmental and evolutionary change. Given this situation, a crucial question within Big Data science concerns how one can bring such different data types, coming from a variety of sources, under the same umbrella.

To address this question, my research over the last eight years has focused on documenting and analysing the ways in which biological data – and particularly ‘omics’ data, the quintessential form of ‘Big Data’ in the life sciences – travel across research contexts, and the significant conceptual and material scaffolding used by researchers to achieve this. For the purposes of this article, I shall now focus on one case of Big Data handling in biology, which is arguably among the most sophisticated and successful attempts made to integrate vast quantities of data of different types within this field for the purposes of advancing future knowledge production. This is the development of model organism databases between 2000 and 2010. ⁴ These databases were built with the immediate goal of storing and disseminating genomic data in a formalised manner, and the long-term vision of (1) incorporating and integrating any data available on the biology of the organism in question within a single resource, including data on physiology, metabolism and even morphology; (2) allowing and promoting cooperation with other community databases so that the available datasets would eventually be comparable across species; and (3) gathering information about laboratories working on each organism and the associated experimental protocols, materials and instruments, thus providing a platform for community building. Particularly useful and rich examples include FlyBase, dedicated to D. melanogaster; WormBase, focused on C. elegans; and The Arabidopsis Information Resource, gathering data on A. thaliana. At the turn of the 21st century, these were arguably among most sophisticated community databases within biology. They have played a particularly significant role in the development of online data infrastructures in this area and continue to serve as reference points for the construction of other databases to this day (Leonelli and Ankeny, 2012). They therefore represent a good instance of infrastructure explicitly set up to support and promote Big Data research in experimental biology.

In order to analyse how these databases enable data journeys, I will distinguish between three stages of data travel, and briefly describe the extent to which database curators are involved in their realisation.

What does it take to assemble Big Data? Implications for Big Data claims

The above analysis, however brief, clearly points to the huge amount of manual labour involved in developing databases for the purpose of assembling Big Data and making it possible to integrate and analyse them; and to the many unresolved challenges and failures plaguing that process.

I have shown how curators have a strong influence on all three stages of data journeys via model organism databases. They are tasked with selecting, formatting and classifying data so as to mediate among the multiple standards and needs of the disparate epistemic communities involved in biological research. They also play a key role in devising and adding metadata, including information about experimental protocols and relevant materials, without which it would be impossible for database users to gauge the reliability and significance of the data therein. All these activities require large amounts of funding for manual curation, which is mostly unavailable even in areas as successful as model organism biology. They also require the support and co-operation of the broader biological community, which is however also rare due to the pressures and credit systems to which experimental biologists are subjected. Activities such as data donation and participation in data curation are not currently rewarded within the academic system. Therefore, many scientists who run large laboratories and are responsible for their scientific success perceive these activities as an inexcusable waste of time, despite being aware of their scientific importance in fostering Big Data science.

We thus are confronted with a situation in which (1) there is still a large gap between the opportunities offered by cutting-edge technologies for data dissemination and the realities of biological data production and re-use; (2) adequate funding to support and develop online databases is lacking, which greatly limits curators’ ability to make data travel; and (3) data donation and incorporation into databases is very limited, which means that only a very small part of the data produced within biology actually get to be assembled into Big Data collections. Hence, Big Data collections in biology could be viewed as very small indeed, compared to the quantity and variety of data actually produced within this area of research. Even more problematically, such data collections tend to be extremely partial in the data that they include and make visible. Despite curators’ best efforts, model organism databases mostly display the outputs of rich, English-speaking labs within visible and highly reputed research traditions, which deal with ‘tractable’ data formats. The incorporation of data produced by poor or unfashionable labs, whether in developed or developing countries, is very low – also because scientists working in those conditions have an even lesser chance than scientists working in prestigious locations to be able to contribute to the development of databases in the first place (the digital divide is alive and well in Big Data science, though taking on a new form).

A possible moral to be drawn from this situation is that what counts as data in the first place should be defined by the nature of their journeys. According to this view, data are whatever can be fitted into highly visible databases; and results that are hard to disseminate in this way do not count as data at all, since they are not widely accessible. I regard this view as empirically unwarranted, as it is clear from my research that there are many more results produced within the life sciences which biologists are happy to call and use as data; and that what biologists consider to be data does depend on their availability for scrutiny (it has to be possible to circulate them to at least some peers who can assess their usefulness as evidence), but not necessarily on the extent to which they are publicly available – in other words, data disseminated through paper or by email can have as much weight as data disseminated through online databases. Despite these obvious problems, however, the increasing prominence of databases as supposedly comprehensive sources of information may well lead some scientists to use them as benchmarks for what counts as data in a specific area of investigation. This tendency is reinforced by wider political and economic forces, such as governments, corporations and funding bodies, for whom the prospect of assembling centralised repositories for all available evidence on any given topics constitutes a powerful draw (Leonelli, 2013).

How do these findings compare to the claims made by Mayer-Schönberger and Cukier? For a start, I think that they cause problems to both of the corollaries to their views that I listed above. Consider first the question of sampling. Rather than disappearing as a scientific concern, looking at the ways in which data travel in biology highlights the ever-growing significance of sampling methods. Big Data that is made available through databases for future analysis turns out to represent highly selected phenomena, materials and contributions, to the exclusion of the majority of biological work. What is worse, this selection is not the result of scientific choices, which can therefore be taken into account when analysing the data. Rather, it is the serendipitous result of social, political, economic and technical factors, which determines which data get to travel in ways that are non-transparent and hard to reconstruct by biologists at the receiving end. A full account of factors involved here far transcends the scope of this article. ⁵ Still, even my brief analysis of data journeys illustrates how they depend on issues as diverse as national data donation policies (including privacy laws, in the case of biomedical data); the good-will and resources of specific data producers, as well as the ethos and visibility of the scientific traditions and environments in which they work (for instance, biologists working for private industries may not be allowed to publicly disclose their data); and the availability of well-curated databases, which in turn depends on the visibility and value placed upon them (and the data types therein) by government or relevant public/private funders. Assuming that Big Data does away with the need to consider sampling is highly problematic in such a situation. Unless the scientific system finds a way to improve the inclusivity of biological databases, they will continue to incorporate partial datasets that nevertheless play a significant role in shaping future research, thus encouraging an inherently conservative and irrational system.

This partiality also speaks to the issue of bias in research, which Mayer-Schönberger and Cukier insist can potentially be superseded in the case of Big Data science. The ways in which Big Data is assembled for further analysis clearly introduce numerous biases related to methods for data collection, storage, dissemination and visualisation. This feature is recognised by Mayer-Schönberger and Cukier, who indeed point to the fact that the scale of such data collection takes focus away from the singularity of data points: the ways in which datasets are arranged, selected, visualised and analysed become crucial to which trends and patterns emerge. However, they assume that the diversity and variability of data thus collected will be enough to counter the bias incorporated in each of these sources. In other words, Big Data is self-correcting by virtue of its very unevenness, which makes it probable that incorrect or inaccurate data are rooted out of the system because of their incongruence with other data sources. I think that my arguments about the inherent imbalances in the types and sources of data assembled within big biology casts some doubt as to whether such data collections, no matter how large, are diverse enough to counter bias in their sources. If all data sources share more or less the same biases (for instance, they all rely on microarrays produced with the same machines), there is also the chance that bias will be amplified, rather than reduced, through such Big Data.

These considerations do not make Mayer-Schönberger and Cukier’s claims about the power of Big Data completely implausible, but they certainly dent the idea that Big Data is revolutionising biological research. The availability of large datasets does of course make a difference, as advertised for instance in the Fourth Paradigm volume issued by Microsoft to promote the power of data-intensive strategies (Hey et al., 2009). And yet, as I stressed above, having a lot of data is not the same as having all of them; and cultivating such an illusion of completeness is a very risky and potentially misleading strategy within biology – as most researchers whom I have interviewed over the last few years pointed out to me. The idea that the advent of Big Data lessens the value of accurate measurements also does not seem to fit these findings. Most sciences work at a level of sophistication in which one small error can have very serious consequences (the blatant example being engineering). The constant worry about the accuracy and reliability of data is reflected in the care employed by database curators in enabling database users to assess such properties; and in the importance given by users themselves to evaluating the quality of data found on the internet. Indeed, databases are often valued because they provide means to triangulate findings coming from different sources, so as to improve the accuracy of measurement and determine which data are most reliable. Although they may often fail to do so, as I just discussed, the very fact that this is a valued feature of databases makes the claim that ‘messiness’ triumphs over accuracy look rather shaky. Finally, considering data journeys prompts second thoughts about the supposed primacy of correlations over causal explanations. Big Data certainly does enable scientists to spot patterns and trends in new ways, which in turn constitutes an enormous boost to research. At the same time, biologists are rarely happy with such correlations, and instead use them as heuristics that shape the direction of research without necessarily constituting a discovery in itself. Being able to predict how an organism or ecosystem may behave is of huge importance, particularly within fields such as biomedicine or environmental science; and yet, within experimental biology, the ability to explain why certain behaviour obtains is still very highly valued – arguably over and above the ability to relate two traits to each other. ⁶

Conclusion: An alternative approach to Big Data science

In closing my discussion, I not only want to consider its specificity with respect to other parts of Big Data science but also the general lessons that may be drawn from such a case study. Biology, and particularly the study of model organisms, represents a field where data have been produced long before the advent of computing and many data types are still generated in ways that are not digital, but rather rely on physical and localised interactions between one or more investigators and a given organic sample. Accordingly, biological data on model organisms are heterogeneous both in their content and in their format; are curated and re-purposed to address the needs of highly disparate and fragmented epistemic communities; and present curators with specific challenges to do with the wish to faithfully capture and represent complex, diverse and evolving organismal structures and behaviours. Readers with experience in other forms of Big Data may well be dealing with cases where both data and their prospective users are much more homogeneous, which means that their travel is less contested and tends to be curated and institutionalised in completely different ways. I view the fact that my study bears no obvious similarities to other areas of Big Data use as a strength of my approach, which indeed constitutes an invitation to disaggregate the notion of Big Data science as a homogeneous whole and instead pay attention to its specific manifestations across different contexts. At the same time, I maintain that a close examination of specialised areas can still yield general lessons, at the very least by drawing attention to aspects that need to be critically scrutinised in all instances of Big Data handling. These include, for instance, the extent to which data are – and need to be – curated before being assembled into common repositories; the decisions and investments involved in selecting data for travel, and their implications for which data get to be circulated in the first place; and the representativeness of data assembled under the heading of ‘Big Data’ with respect to other (and/or pre-existing) data collection activities within the same field.

At the most general level, my analysis can be used to argue that characterisations of Big Data science as comprehensive and intrinsically unbiased can be misleading rather than helpful in shaping scientific as well as public perceptions of the features, opportunities and dangers associated with data-intensive research. If one admits the plausibility of this position, then how can one better understand current developments? I here want to defend the idea that Big Data science has specific epistemological and methodological characteristics, and yet that it does not constitute a new epistemology for biology. Its strength lies in the combination of concerns that have long featured in biological research with opportunities opened up by novel communication technologies, as well as the political and economic climate in which scientific research is currently embedded. Big Data brings new salience to aspects of scientific practice which have always been vital to successful empirical research, and yet have often been overlooked by policy-makers, funders, publishers, philosophers of science and even scientists themselves, who in the past have tended to evaluate what counts as ‘good science’ in terms of its products (e.g. new claims about phenomena or technologies for intervention in the world) rather than in terms of the processes through which such results are eventually achieved. These aspects include the processes involved in valuing data as a key scientific resource; situating data in a context within which they can be interpreted reliably; and structuring scientific institutions and credit mechanisms so that data dissemination is supported and regulated in ways that are conducive to the advancement of both science and society.

More specifically, I want to argue that the novelty of Big Data science can be located in two key shifts characterising scientific practices over the last two decades. First is the new prominence attributed to data as commodities with high scientific, economic, political and social value (Leonelli, 2013). This has resulted in the acknowledgement of data as key scientific components, outputs in their own right that need to be widely disseminated (for instance, through so-called ‘data journals’ or repositories such as Figshare or more specialised databases) – which in turn is engendering significant shifts in the ways in which research is organised and assessed both within and beyond scientific institutions. Second is the emergence of a new set of methods, infrastructures and skills to handle (format, disseminate, retrieve, model and interpret) data. Hilgartner (1995) spoke about the introduction of computing and internet technologies in biology as a change of communication regime. Indeed, my analysis has emphasised how the introduction of tools such as databases, and the related opportunity to make data instantly available over the internet, is challenging the ways in which data are produced and disseminated, as well as the types of expertise relevant to analysing such data (which now needs to include computing and curatorial skills, in addition to more traditional statistical and modelling abilities).

When seen through this lens, data quantity can indeed be said to make a difference to biology, but in ways that are not as revolutionary as many Big Data advocates would advocate. There is strong continuity with practices of large data collection and assemblage conducted since the early modern period; and the core methods and epistemic problems of biological research, including exploratory experimentation, sampling and the search for causal mechanisms, remain crucial parts of inquiry in this area of science – particularly given the challenges encountered in developing and applying curatorial standards for data other than the high-throughput results of ‘omics’ approaches. Nevertheless, the novel recognition of the relevance of data as a research output, and the use of technologies that greatly facilitate their dissemination and re-use, provide an opportunity for all areas in biology to reinvent the exchange of scientific results and create new forms of inference and collaboration.

I end this article by suggesting a provocative explanation for what I argued is a non-revolutionary role of Big Data in biology. It seems to me that my scepticism arises because of my choice of domain, which is much narrower than Mayer-Schönberger and Cukier’s commentary on the impacts of Big Data on society as a whole. Indeed, biological research may be the domain of human activity that is least affected by the emergence of Big Data and related technologies today. This is precisely because, like many other natural sciences, such as astronomy, climatology and geology, biology has a long history of engaging with large datasets; and because deepening our current understanding of the world continues to be one of the key goals of inquiry in all areas of scientific investigation. While often striving to take advantage of any available tool for the investigation of the world and produce findings of use to society, biologists are not typically content with establishing correlations. The quest for causal explanations, often involving detailed descriptions of the mechanisms and laws at play in any given situation, is not likely to lose its appeal any time soon. Whether or not it is plausible in its implementation, the Big Data epistemology outlined by Mayer-Schönberger and Cukier is thus unlikely to prove attractive to biologists, for whom correlations are typically but a starting point to a scientific investigation; and the same argument may well apply to other areas of the natural sciences. ⁷ The real revolution seems more likely to centre on other areas of social life, particularly economics and politics, where the widespread use of patterns extracted from large datasets as evidence for decision-making is a relatively recent phenomenon. It is no coincidence that most of the examples given by Mayer-Schönberger and Cukier come from the industrial world, and particularly globalised sales strategies, as in the case of Amazon.com. Big Data provides new opportunities for managing goods and resources, which may be exploited to reflect and engage individual preferences and desires. By the same token, Big Data also provides as yet unexplored opportunities for manipulating and controlling individuals and communities on a large scale – a process that Raley (2013) characterised as ‘dataveillence’. As demonstrated by the history of quantification techniques as surveillance and monitoring tools (Porter, 1995), data have long functioned as a way to quantify one’s actions and monitor others. ‘Bigness’ in data production, availability and use thus needs to be contextualised and questioned as a political-economic phenomenon as much as a technical one (Davies et al., 2013).