Governing-by-the-numbers/Statistical governance: Reflections on the future of official statistics in a digital and globalised society

1. Introduction

“Data meet politics”, under this cover, well-known guiding themes, such as the modernisation of the public sector, or evidence-informed policy-making, are led to new solutions with new technologies and infinitely rich data sources by “using 21st century tools to address 21st century issues.” The possibilities for “cutting edge solutions”, “close to real time data” seem obvious if we succeed in using “advanced analytics, both timely and simple and clear enough for fast-paced policy decisions”.2 The question arises why, despite all these modern and innovative possibilities, there should still be a need for elaborate official statistics in the future. Why should a politician wait for official statistics when there was real-time data for the most difficult decisions? Why afford a statistical service to the cost of taxpayers when it seems so easy to analyse the data that all the machines mutually connected within the Internet of Things are generating all the time? As can be seen quickly, Data for Policy means more to official statistics than just new data, techniques and methods. It is not least a matter of securing an important function and position for official statistics in the Policy for Data of the future. In order to justify this position, it is necessary to have a clear understanding of the tasks of official statistics for the functioning of (democratic) societies, with a view to how these tasks have to be reinterpreted under changing conditions (above all because of digitisation and globalisation).

One might now come to the conclusion that these are all questions and topics with which the various actors in statistics have to come to terms, a purely practical matter. Why then do we need an article in a scientific journal? However, as will be explained below, it is very much a subject of scientific research, and not necessarily of those that are already being addressed in statistics, namely questions of survey methodology, sampling, error correction, etc. Rather, the scientific inputs in this field also and especially come from other disciplines, such as sociology or political science.

Furthermore, one may have the impression that the issues dealt with in the following, which deal with information quality, interaction with users and statistical governance, are relevant only to a small group of statisticians, especially those responsible for the strategic direction of official statistics. This impression is also misleading. The very basic topics, such as epistemology, form the foundation for a correctly understood and applied statistical methodology. In this respect, they belong not least to the elementary programme of statistical education and professional literacy.

The thesis is put forward that the questions and topics dealt with in this article are currently covered in many ways by methodological developments, scientific investigations and practical tests, but that these take place in partly very different communities and that mutual fertilisation is insufficient. For example, the Science and Technology Studies located in sociology or the discourse on Governing by the Numbers are little or not at all known in statistical communities. In mirror image, in scientific circles, for example in sociology, the knowledge and awareness of what quality standards have been implemented in official statistics in recent years is insufficient.

Especially under the current conditions of digitalisation, globalisation and the increasingly widespread scepticism towards experts and facts, it is therefore important to close the gaps between the discussions in different communities.

2. From evidence-based to data-driven decision-making

Since the 1990s at the latest, it has been part of the basic understanding of a modern and good governance that politics is based on evidence [2, 3]. Irrespective of the political colour of the rulers, a modernisation of the administration according to the New Public Management model was promoted. Rational, pertinent, legitimate and accountable decision-making requires that all available information be taken into account. If decisions are not supported by evidence, there are many dangers such as lack of transparency, subjectivity or even erroneous policies. In this respect, the demand for evidence-based policy-making (EBPM) is a tautology in times of enlightened, democratic societies that request efficiency. The origins and sources of EBPM can therefore also be found in the approaches to a rational and economic approach, be it in management theories at the enterprise level or in economic theory for politics.

More recently, the use of language has evolved to the extent that we are now talking about ‘data-driven policy-making’ [4] or more simply ‘data-for-policy (D4P)’ [5]. The background to this is the enormously rapid increase in the amount of data that should be used – provided the appropriate technologies and methodologies are in place – to further improve public administration in general and political decisions in particular.

Although this more recent orientation must be understood as a logical and consistent further development, it contains new elements and sets new priorities. First, the term ‘data’ takes the place of ‘evidence’, which is derived from the fact that data is regarded as the new fuel of the 21st century. Organisations and administrations should be data-centred, and decisions should be data-driven accordingly. In combination with algorithms, machine learning (so-called artificial intelligence) and the Internet of Things, which all rely on these data and use them as fuel, this results in a new dimension of decision making; instead of augmenting decisions, they are becoming entirely automated.

This makes it obvious that the question of the interaction of technologies, of political/social developments and of data/information (as tools of power and governance) in the era of digitalisation arises in a completely new and intensified form. Paradoxically, data/ information can also be misused and lead to the opposite of what was originally intended with EBPM, in a similar form to the Internet, which was supposed to promote democracy and also gives birth to authoritarian applications [6].

Of course, the developments described are of a general nature and affect all areas of the economy, public life and politics. However, the sole aim here is to show the effects on official statistics and possibly to draw conclusions for necessary measures and strategic directions. Official statistics in this sense is understood as the part of the public administration entrusted with providing society with solid statistics on the essential topics of population, social or environmental issues, and economy. As the more than two-hundred-year history of official statistics shows, there were not only steady developments, but also abrupt leaps in development, especially when technological, scientific and political driving forces were mutually reinforcing [7, 8]. Just as there is talk of a fourth industrial revolution, official statistics can also be seen in the midst of a period of rapid and fundamental change, in which raw materials (i.e. data), production processes and user expectations are shifting dramatically.

In this situation it is indeed possible to react with the adaptation of methods and processes to modified conditions in the usual continuous pattern. However, this is unlikely to be sufficient. Rather, in a period of fundamental societal and technological changes, it seems necessary to become clear again about the essence of what official statistics have been in the past, what they are today and what they should be in the future.

A crucial question in this context is whether the production of evidence on the one hand and policy decisions on the other are separate and independent processes. As can be seen from the many, sometimes dramatic examples (e.g. the start of the financial crisis 2009 through falsified Greek statistics, Brexit, to name but a few) [9, 10], this independence cannot be simply assumed. On the contrary, for the adequate use of EBPM or of D4P, it is essential to know the risks and side effects resulting from the fact that the processes of quantification, measuring and decision-making are systematically and permanently interrelated.3 Based on the sociological concept of ‘co-production’ from the Science and Technology Studies (STS), the different effects, feedback and pitfalls can be systematised and analysed. Particularly against the background of increasing uncertainty and scepticism towards experts in society, it is crucial that these experts and participants in political decision-making processes are conscious of the risks and side effects, are aware of their importance and take them into account when dosing and monitoring the efficacy of the ‘medicine’ called EBPM/D4P.

At the centre of this strategically important topic for official statistics is therefore the interrelation between, on the one hand, ‘Governing-by-the Numbers’ [11] and, on the other hand, ‘Informational Governance’ [12].

Some quite fundamental questions will have to be asked and discussed in the following:

What are facts (also including related terms, such as data, information, evidence)?

3. What are facts?

As a statistician, you are often surprised by the peculiar expectations or comments you are confronted with:

On the one hand, there is an almost blind faith in the existence of facts as an absolute form of truth. “Look at the data!” “Let’s do a fact check!” “You can’t manage what you don’t measure!” belong to such widespread and popular mantras not seldom accompanied by a good portion of naivety.

Due to a lack of statistical literacy and missing information about the quality of statistics, a dangerous mixture of feelings of dependence and distrust seems to emerge. Statistics are forced into the dichotomy of truth and lies, which are not only unsuitable, but also stand in the way of a proper handling of statistical information, based on a knowledge of its actual possibilities and limitations. More recently, it appears that this situation is not improving but worsening. There is a debate about facts and alternative facts, about news and fake news, without the core question being asked and answered in public what facts actually are.

In his book “Postfaktisch” Vincent F. Hendricks [13] presents a scale of information quality “in which true and different forms of false statements and strategies undermining truth face each other at opposite ends”. While he goes into detail on the various variants of misinformation, such as distorted statements, lies, fake news, it remains unclear what he understands by ‘true statements’. As a definition he offers “Verified Facts”. Irrespective of whether it makes sense and is helpful to deal with the concept of truth in this context, from the statistician’s point of view the question arises as to what facts are and how they should and can be verified.

It should be noted that statistical facts are the end product of processes that begin with the design of a methodology (translation of a question into a quantifiable variable, definition of the survey programme, etc.), which are secondly produced according to this methodological design, and which are finally communicated to those who wish to use the information for their respective questions. In other words, facts are manufactured products. Like other products, they can have a good design that functionally performs what is expected. As with other products, manufacturing errors can occur. As with other products, misunderstandings and errors can occur during delivery (here the communication). Due to this nature of statistical information, emotionally overloaded questions about truth and lies no longer arise. Rather, it is a matter of producing quality and communicating this quality in such a way that the users of the information can understand this and draw the right conclusions for themselves. However, all these are tasks and problems that occur in more or less the same form for all (industrial) products (that the user has not produced him/herself). Questions of quality management, transparency, labelling and certification naturally play a major role in this context.

These remarks are in danger of being misunderstood to mean that a kind of traditional deductive approach, which is supposedly the only one used in official statistics, is advocated. It should therefore be made clear that the tradition of official statistics is characterised by an interplay between theoretical model and empirical data evaluation, between deductive and inductive procedures [14]. This interplay is embedded in an iterative learning process that leads year after year to adjustments to new circumstances (be it information needs or data bases or methods). At the end of the day, however, official statistics are assessed by their compliance with international standards (which are themselves regularly revised), their comparability in terms of time and space, etc. An inductive reasoning (“data first”) to generate new questions will therefore not become the typical and frequently used approach in official statistics in the future either. Nevertheless, even more often than in the past, the use of secondary data could generate new questions and theories which could then be confirmed by deductive methods and models and eventually lead to statistical facts. This is an approach, labelled experimental statistics,7 which is currently pursued by official statistics.

Three dimensions are decisive for the quality of statistical information: first, statistical measurement quality; second, theoretical-methodological consistency; and third, relevance for information needs and decisions. Only if all three aspects are achieved satisfactorily, or better ’adequately’, can a statistical number, indicator, graph or map play its role: Because only then it is fit for purpose. The history of the last two hundred years also confirms that it were the forces behind these three spheres of influence (science, society, statistics) that drove the development of official statistics.

Finally, a look at the statistical community itself with the same questions: What are facts? What about the knowledge of the epistemological basics of statistics? Here it turns out that in the circle of professional statisticians reflexive approaches and the understanding of the product characteristics of statistical information are only of rather limited popularity.8 An awareness that the “map is not the territory”9 [15] and that this fundamentally shapes the view of the character, quality profile and purpose of statistics is not very pronounced. For example, the technical term “ground truth” is used for a kind of objective ( = true) data, i.e. data derived from surveys, even though such objectivity does not exist in the sense of capturing reality itself. Similarly unreflected (or one may say naïve) is the view that data speak for themselves; as if it were sufficient to search through the basic data with an intelligent algorithm in order to extract the diamonds of informational signals from it.10 In both cases, the task of statistics (and in particular official statistics) is reduced to a purely technical process in the course of which knowledge of appropriate mathematics and IT is sufficient. The various interactions between production and the use of statistics are ignored and the crucial questions of information quality in particular related to relevance, consistency and coherence of information are left untouched.

First findings can be summarised here:

Basic statistical training should be grounded on a critical reflection of the subject of statistical information itself. What are facts? If this is clear, the concepts of quality management, communication of quality and a broad improvement of statistical literacy can be approached. Statistical competence and literacy must not be reduced to mathematical-technical skills. Rather, it also requires an understanding of the fundamentals of the humanities [17].

4. From data to policy and retour

After taking this step and characterising statistics as products, we should now analyse its manufacturing method and the process chain through to delivery and use of the end products. If it is a long way from crude oil to the resulting intermediate and end products and the most diverse applications and uses, why should it be different in terms of data, information, knowledge and application? It is only when one understands that on this long journey the most diverse professions and expertise are required, which cooperate one after the other and with each other, that one is able to deal appropriately with the topic of quality.

The growing importance of statistical evidence, data and information for political decisions is reflected in the handy and popular formulation ‘Data for Policy’ (D4P).11 However, this label is only suitable to a limited extent for characterising the network of relationships and mutual influence between data on the one hand and politics on the other. Even if the amount of data available to inform policy-making (including organisational, administrative decision making in a more general sense) is growing at an enormous rate in the era of digitalisation and globalisation, this raw material of data is not directly usable in politics. Hence, tailored processes are needed to distil, refine and process valuable statistical knowledge from the flood of raw data into digestible information for politics. The term ‘facts’ is used here as a generic term for such information. While statistical data represents the beginning of the processing, facts characterise its end.

With this distinction between data and facts, it is possible to break down the relationship between policy and data into different components that inform policy in different ways (see Fig. 1).

OPEN IN VIEWER

Within this broader relationship, the following fields must be distinguished from each other:12

Data to facts (D2F): This is the outcome of data analysis, in which statisticians, data scientists, and empirically working researchers/analysts from different disciplines engage. Such facts can be the result of standard statistical processes on the one hand, but on the other hand also, for example, unique research-based evaluations of microdata.

Facts to policy/politics (F2P): This is the field of work of specialists who prepare and use the information content of facts for policy advice. Journalists, researchers and policy analysts focussing on future-oriented models are among them, just as policy-makers engaged in evidence-informed politics.

Policy/politics to facts (P2F): On the one hand, this is about the statistical design (i.e. conceptual form, choice of variables, determination of the work programme etc.) for matters that are of importance to society and are supposed to represent the general information needs; those aspired facts are the outputs of the statistical process in form of indicators, accounts, indices, maps, graphs, etc. On the other hand, it also includes questions of knowledge creation governance (who participates in the design process, who ultimately decides on the selection of the statistical programme, how much money and time is available, etc.).

Facts to data (F2D): This is the scientific and technical conception of the generation or selection of suitable data sources (inputs) to quantify facts (output); questions of authorisation, confidentiality, accessibility and ownership of the data are included.

Policy/politics to data (P2D): In many ways,evidence-informed policies necessitate new data and politics sets the framework conditions for the generation of such new data, including for their protection, for the infrastructure of research institutions or data centres, for the design of legal conditions, such as copyrights. Politics also influences the economic framework conditions under which industries develop innovatively and competitively (or not) in the digital era.

Data to policy/politics (D2P): New data and innovative methods of data-science can be used for experimental statistics which can be helpful in the policy-making process to guide priorities and identify future policy issues. Such a role of “big” data and data scientists as an institution of analysis directly and immediately delivering to politics undoubtedly has its limits and risks. Appropriate governance (reporting, committee work, etc.) for sense-making of such new statistical knowledge needs to be established; complementarity with more standardised statistical production methods needs to be emphasised.

Official statistics fulfil an essential duty for the functioning of democratic societies. As in the past, statistics will also be utilised for the making of the future state(s), may it be for good or bad. As in the past, official statistics will continue to be required to reconcile three fundamental rights: the right to privacy of individual data, the right of access to information and the right to live in a civilized society provided with sound information. This is a great responsibility that official statistics must live up to by reflecting their production methods scientifically and at the same time fighting for their appropriate place in the informational ecosystem of globalised societies.

The label ‘Data for Policy’ is only suitable to a limited extent for characterising the network of relationships and mutual influence between data on the one hand and politics on the other. Even if the amount of data available to inform decision-making is growing at an enormous rate in the era of digitalisation and globalisation, this raw material of data is not directly usable in politics. Hence, tailored processes are needed to distil, refine and process valuable statistical knowledge from the flood of raw data into digestible information for politics.

Data are given, facts are made. While data represent the beginning of the processing, facts characterise its end. Terminology must be unambiguous; it must be chosen professionally as well as consciously and has great importance in communication. Unfortunately, it is common today to use ‘data’ very widely, without distinguishing between raw material (i.e. the data) and end product (i.e. the facts). This equalisation creates confusion and makes communication of information quality difficult. The worst thing, however, is that this fuzziness is detrimental to the profile of statistics in the informational ecosystem.

Basic statistical training should be grounded on a critical reflection of the subject of statistical information itself. What are facts? If (and only if) the answer to this question is clear, the concepts of quality management, communication of quality and a broad improvement of statistical literacy can be approached. Statistical competence and literacy must not be reduced to mathematical-technical skills. Rather, it also requires an understanding of the fundamentals of the humanities.

Quality of statistical information can be achieved through total quality management, which includes all production processes as well as professional marketing. Management of quality requires one thing most of all: quality of management.

Scientific research and development are essential to the quality of statistics, whether they are based on survey methodologies, or driven by data science concepts. Apparently, this relates in the first place to the relevant technical disciplines. However, it is equally necessary to take on board aspects from other fields, such as sociology, historical, or legal disciplines. There are many different strands of science contributing research on processes of quantification, and the impact of quantification within social contexts.

Participation of civil society needs to be anchored in the working culture of the statistical factory. The strategic goal here is to intensify the partnership between civil society and statistics in all the stages of the manufacturing: in the scientific and design phase, during production, and – most importantly – through communication.

The power of numbers will increase dynamically with new data sources and technologies. Data for policy call for a comprehensive policy for data at both national and especially international level. Official statistics can and must claim a decisive role in this Policy for Data. Building on existing legal and codified principles and rules, further developments should be actively pursued to meet the new challenges. A global organisation of professional statistics anchored in civil society should monitor the independence and integrity of statistics in individual countries and develop a suitable indicator.

The development of official statistics has been and continues to be influenced by new data, by new methods or by society’s new information needs. We are now seeing all three driving forces changing very rapidly, even dramatically. With a forward-looking strategy, official statistics should seize the existing opportunities to remain what they are: policy relevant, but not politically driven.