Abstract
National governments are increasingly aware of the importance of carrying out Open Government Data (OGD) initiatives for the provision of public sector information and for use and reuse without restriction or charge by society. Therefore, the aim of this article is to examine and compare the quality of the OGD portals created by national governments, in terms of three analytical aspects: function, semantics, and content, and to identify the impact of diverse factors on the quality of these portals. The results obtained show there is considerable heterogeneity among OGD portals, reflecting the lack of harmonization and the need for quality standards. In this regard, the quality of the OGD portals examined is only at an intermediate level and is affected by the level of Internet penetration and the economic capacity of each country.
Introduction
In recent years, governments around the world have developed Open Government Data (OGD) initiatives, understood as the combination of open government and open data. The open government concept implies greater transparency, participation, accountability, and access to public information (Sonáglio, 2013), whereas open data refer to the dissemination of information that can be freely used, reused and redistributed, at no charge (Bauer & Kaltenböck, 2011).
OGD are provided by Internet portals through which public sector information (PSI) is made available for use by different stakeholders (Alonso et al., 2009). This provides an environment favoring proactive citizen participation, such as allowing users to create applications from the data made available (Both & Schieferdecker, 2012), thus encouraging information feedback between society and government (Kassen, 2013). PSI can be available in two types of formats, human-readable in order to add value to simple users (i.e., pdf file) and machine-processable to add value to advanced users (i.e., JavaScript Object Notation (JSON) file).
The OGD movement includes initiatives produced by different levels of government and diverse public agencies. Kalampokis, Tambouris, and Tarabanis (2011) note that these initiatives have been established in all governance level, federal, national, regional, and local level. As a result of these initiatives, in 2009, the United States adopted the Open Government Directive (Orszag, 2009) and the U.K. government issued the document “Power of Information Taskforce Report” (HM Government, 2009). Moreover, the G8 leaders have become involved in approving the Open Data Charter in 2013, and international platforms, such as the Open Government Partnership (OGP), have been created in this respect (OGP, 2014).
However, as noted by Parycek, Hochtl, and Ginner (2014), most countries are subject to regulations or codes that do not specify minimum requirements concerning OGD policies and portals. A study conducted by the Spanish Ministry of Finance and Public Administration in 2012 showed that diverse stakeholders have called for greater coordination in this matter to ensure the public and timely availability of the same types of information. Some initiatives in this respect have been taken, such as the program undertaken in 2014, to harmonize the OGD portals of Mediterranean countries through the HOMER Project, the main aim of which is to establish a strategy to harmonize the policies and the level of quality of OGD portals in these countries (HOMER, 2014).
The principle of promoting the analysis and reuse of OGD data is a primary concern with respect to OGD quality (Tauberer, 2014). Also, the quality of OGD portals is a factor that can affect users’ ability to locate the data of interest (Kučera, Chlapek, & Nečaský, 2013). In this sense, recent experience shows that OGD portal quality requires careful attention by designers and providers (Kučera et al., 2013). Moreover, Batini, Cappiello, Francalanci, and Maurino (2009) suggest that an OGD catalogue should be characterized to meet certain quality requirements, such as accuracy, completeness, consistency, and timeliness. Nevertheless, as observed by Fitzpatrick (2012), so far very little research has been conducted in this area, and the few studies published have focused on the quality characteristics intrinsic to making data available (Berners-Lee, 2010; Cyganiak, 2011).
With the exception of the United States and some local government portals, most OGD initiatives do not address the question of OGD quality (Both & Schieferdecker, 2012; Lakhani, Austin, & Yi, 2010). Similarly, the European 2003/98/EC Directive (European Community [EC], 2003) and 2013/37/UE Directive (European Parliament, 2013) fail to specify any quality levels for the data provided. In the absence of a common guidance on how OGD portals should be structured, each government may establish the quality of its website in a different way. Therefore, it would be desirable to determine the current outlook of different governments regarding the availability of PSI and the possibilities for its reuse.
Taking into account the above considerations, the present study has a twofold aim. On the one hand, to analyze and compare the quality presented by OGD portals in countries that have a platform of this type at a national level, conducting our analysis by means of an index of quality. On the other hand, to identify the variables that may influence the levels of quality achieved by governments on their OGD portals.
The findings of this study could bring about an increased understanding of the benefits offered by high-quality OGD portals and provide guidance to public agencies seeking to use them to their full potential, maximizing the reuse of PSI by different stakeholders. In addition, we hope this research will respond appropriately to the need to create a common framework for access to public data and for its reuse, from a functional, semantic, and content-oriented standpoint.
Method
Taking into account the two main aims of this study, a descriptive analysis was made of the quality presented by the OGD portals of countries that have such a platform available nationwide. We then conducted an explanatory analysis to determine the variables that might influence the quality levels of governmental OGD portals.
Descriptive Analysis
In recent years, many authors have proposed models with which to measure web portal quality, regardless of the sector in which it operates (Hasan & Abuelrub, 2011; Polillo, 2012). However, as regards the quality of OGD portals—and these too are web portals—this is a novel research area, and, so far, one that is virtually unexplored. Authors such as Kučera, Chlapek, and Nečaský (2013) and Tauberer (2014) have examined the quality of OGD portals from different perspectives but without establishing a quality model, merely analyzing different dimensions of these portals, such as accessibility, data quality, or linking.
On the other hand, Alexopoulos, Spiliotopoulou, and Charalabidis (2013) focused on analyzing OGD portals using functional, semantic, and technical measurement practices. In this line, too, Kalampokis et al. (2011) proposed a classification scheme for different OGD initiatives based on the level of government (federal, national, regional, and local). Other studies have sought to establish standards of openness and data availability in OGD portals, as in the case of the World Wide Web Consortium (W3C, 2009), or have proposed a model of maturity to measure the ways in which data are published according to the characteristics of reuse and data linking in OGD portals (Berners-Lee, 2010). Nevertheless, it can be seen that there is an absence of metrics with which to measure or assess the quality of OGD portals.
Taking into account this previous literature, a quality index (QI) was developed to measure the quality of OGD portals. This index, following Kalampokis et al. (2011), Alexopoulos et al. (2013), and Kučera et al. (2013), distinguishes between the functional, semantic, and content-based aspects of OGD portals, and thus three subindices (functional subindex [FSI], semantic subindex [SSI], and content-based subindex [CSI], respectively) were created. Table 1 shows the items that make up each subindex.
FSI of quality
In line with Alexopoulos et al. (2013) and Kalampokis et al. (2011), we examined the quality of OGD portals from a functional perspective, analyzing the set of utilities and functions with which users can obtain information. This analysis focuses on 5 items: data search techniques, the organizational approach to data provision (direct or indirect), data supply, visualization, and feedback (see Table 1 first section).
SSI of quality
Functional, Semantic, and Content-Based Aspects of OGD Portals.
Note. OGD = Open Government Data; XLS = MS Excel file extension; CSV = comma-separated values; wfs = web feature service; gml = Geography Markup Language; shp = Shapefile; wms = Web Map Service; kml = Keyhole Markup Language; kmz = Keyhole Markup Language Zipped; png = Portable Network Graphics; ZIP = It’s an archive file format that supports data compression; DOC = It’s the Microsoft Word’s native file format; RDF = Resource Description Framework; ods = Open Document Spreadsheet; HOMER = Harmonising Open data in the Mediterranean through better access and Reuse of public sector information.
OGD portal quality was analyzed from a semantic perspective, examining the arrangement of the data through both simple flat structures and complete linked ones. This analysis was based on the method described by Alexopoulos et al. (2013), Berners-Lee (2010), and Interoperability Solutions for European Public Administrations (2011), in which 4 items are distinguished: level of metadata, level of open data, multilingualism, and data format (see Table 1 second section).
CSI of quality
Taking into account Kučera et al. (2013) and other studies that have analyzed website quality (Martín, Hernández, & Jiménez, 2007; Polillo, 2012), we evaluated the relevance of the information provided by countries through their OGD portals, as the provision of information is the basic function of a website. In this regard, Morville and Rosenfeld (2007) argued that a fundamental aspect of quality is that of information architecture and its content. Accordingly, we examined, first, whether the information provided on the OGD portal was accurate, complete, consistent, and timely and, second, the type of information disclosed, the categories into which information was compiled, the filters used to locate information, the total volume of data, and the number of agencies involved. For this purpose, the following 10 items were considered: free information, data accuracy, timeliness, applications, glossary or list of terms, the possibility of incorporating new data, data categories (based on Directive 2003/98/EC on the classification of PSI), volume of data, number of agencies, and types of filters (see Table 1 third section).
The QI was determined by dividing the sum of the scores for all items (QI i ) by the total number of items observed (19 items). To express this as a percentage, the resulting value was multiplied by 100. Each subindex (FSI, SSI, and CSI) was determined by the same procedure, taking into account the number of items in each one.
Explanatory Analysis
Many articles have been published in which OGD portals are analyzed in terms of issues such as their legal basis, the political and economic environment, and the possibilities for organizations to publish data. Some of these studies have examined the worldwide situation of OGD portals (Peset et al., 2011), while others have focused on specific countries (Bertot, Jaeger, & Grimes, 2010; Shadbolt & O’Hara, 2013) or cities (Kassen, 2013).
Regarding the factors that affect OGD portal quality, there has been scant research in this field, too, mainly focusing on websites. In this respect, De Andrés, Lorca, and Martínez (2010) and Lorca, De Andrés, and Martínez (2012) have identified a series of economic and cultural factors that are crucial to the accessibility of corporate sector websites. Other studies have analyzed the factors that determine the dissemination and accessibility of government information in web portals, and chief among these factors are:
Corruption: Taking into account the fact that OGD portals form part of e-government (Bertot et al., 2010), and the impact of e-government on levels of corruption (Rodríguez, García, & Gallego, 2011), we believe it useful to analyze the relationship between corruption and OGD portal quality. Internet penetration: Rodríguez, García, and Gallego (2011) and Caba, Rodríguez, and López (2014) have all examined the relationship between Internet penetration and the quality of information disclosed on the web. Therefore, with respect to OGD portals, it would be appropriate to consider whether there is a relationship between levels of Internet penetration within society and the quality levels achieved by OGD portals. Economic capacity: Authors such as Gandía and Archidona (2008), Rodríguez et al. (2011), and Caba et al. (2014) have analyzed the influence of the economic capacity of the population on the level of development of e-government and on the quality of the information disclosed, finding a significant positive association between these variables. In the case of OGD portals, it would be interesting to analyze whether the country’s economic capacity is a determinant factor in OGD portal quality. Education level: Rodríguez et al. (2011) and Caba et al. (2014) have highlighted the importance of this factor in the adoption of e-government and in the quality of the information disclosed. Therefore, our analysis included the question of the impact of the level of education in a country on the OGD portal quality achieved. Online administration and participation: The development of e-government through e-presence, the delivery of e-services, and the implementation of e-democracy play a determinant role in the evolution of public sector administration. Focusing on e-services, which are used to improve online citizen services, and e-democracy, which is used to promote greater online participation by citizens, and following Rodríguez et al. (2011), we consider it useful to examine the impact of online administration and of online participation on OGD portal quality, as an integral part of e-government.
This study does not attempt to model in detail all factors that affect OGD portals quality. Instead, it focuses on those that are relevant, according to the literature. In any case, there are other interesting factors such as the government size, measured by the number of public sector employees (Rodríguez et al., 2011), the level of civil liberties, defined according to the index “Voice and Accountability” (Kaufmann, Kraay, & Mastruzzi, 2008), or the public media visibility, considering whether or not visibility on the Internet of the government’s activities impacts in the information disseminated through the website of the government (Gandía & Archidona, 2008).
After selecting the explanatory variables that influence the quality of OGD portals, and defining the hypothesis and metric (see Table 2), the correlation between them was determined. The QI was selected as the dependent variable (see Figure 1 conceptual model). We used the QI since it includes all the quality items of the quality subindices. Then, taking into consideration that the dependent variable is censored, taking values between 0 and 1, a Tobit regression analysis was performed.
Determinant Factors of OGD Portal Quality.
Note. OGD = Open Government Data; PISA = Programme for International Student Assessment; GDP = Gross domestic product.
Test of the conceptual model.
The structural equation in the Tobit model is:
Sample
The OGD portals included in this study were selected from the list provided by the Data.gov platform in the “International Countries” category. This platform is the official site of the U.S. OGD and the website of the world’s largest open government (Hendler, Holm, Musialek, & Thomas, 2012). Of the 46 national OGD portals listed by Data.gov, 10 were rejected because their OGD portal did not meet the basic standards for an open data portal. Therefore, the final sample was composed of 36 national OGD portals, 6% of which belonged to countries in Oceania, 44% to European countries, 19% to Asian ones, 17% to North and South American ones, and 14% to African countries. This review of OGD portals was conducted during June–July 2014.
Results
Descriptive Analysis
The QI results show that the average quality of governmental OGD portals does not exceed 50% of recommended levels (see Table 3). However, 21 countries scored above the average. Canada achieved the highest level of quality, with its OGD portal meeting 78% of requirements. By contrast, Finland only met 10% of the quality requirements.
Quality Index Results of OGD Portals by Country.
Note. OGD = Open Government Data The gray shading values are the highest values obtained. The highest values obtained in the Quality Index appear in bold.
Of the three subindices of portal quality, the functional aspects reflected the highest levels of compliance among OGD portals, at an average level of 61%. We analyzed the items that constitute these functional aspects (see Table 1). Starting with data search techniques, 94% of the countries examined present a simple list, together with data search by categories. Most countries also offer the option for free text data search, in which words are entered in order to locate the data required. In this regard, Bahrain provides the entire data search techniques analyzed. In contrast, data search by means of interactive maps is used very little (11%). When information is presented in the form of an interactive map, the portal incorporates a digital tool (with the official maps of the country) to optimize geo-information searching.
As to organization approach, 67% of countries provide information on their OGD portals directly, 22% do so indirectly, and 11% combine these two approaches, as is the case in Canada and Germany.
With respect to data provision, over half of the OGD portals allow data to be viewed online and downloaded, as it is the case of Kenya and Portugal. However, 39% only allow users to download the data, while 6% only allow online viewing. Of the OGD portals where information can be consulted online, it is offered as an image in 61% of the cases, as text in 56%, and as audio in 3%.
The final item related to the functional aspects of OGD portal quality is that of the feedback options provided, and the most common are forms (67%), followed by e-mail (47%) and the telephone (22%).
The results obtained for the semantic aspects of OGD portal quality (SSI) show that, on average, the SSI quality is 51%, with the leading country in this respect being Canada (97% compliance). By contrast, China scored 0% for SSI.
The items that constitute the semantic aspects show that the metadata level is high, with 75% of the countries analyzed using the metadata system in their OGD portals, thus obtaining reusable metadata in 69% of cases. On the other hand, the overall level of open data can be considered acceptable, since almost 80% of these countries use open formats on their OGD portals (*** and ****). Moreover, 39% of the portals are at the highest levels of the classification used (the Berners-Lee Five Stars of Linked Data reduced to four stars), as it is in the case of Canada and Spain.
Only 14% of the countries (including Spain and India) offer the possibility of consulting their OGD portal in a language other than the official one. Furthermore, some countries with more than one official language allow the user to navigate on their OGD portals in any of the official languages.
Of the formats used to present the data, the most widely used is XLS, which is present in 30 OGD portals, followed by comma-separated values (CSV) and Extensible Markup Language (XLM). At the other extreme, the least used formats are those that are least editable, such as image types (GIF and JPEG), and the ods format. Of all the countries analyzed, the largest number of formats is offered by Austria.
The subindex referring to content-based aspects of OGD portal quality (CSI) produced the lowest scores of the three subindices, with an average of 43%. In this regard, Italy with 65% obtained the highest levels of content quality and Finland, with 0%, the lowest.
Our analysis of the items relating to content quality shows that these OGD portals are available free of charge in all cases. The requirement for data accuracy, that is, the data content should match the information described in the catalogue, is met by 94% of these OGD portals. With respect to the timeliness of the data, that is, the data provided on the portal are up to date and not merely historical, as measured by the number of days during which the least up-to-date information is present, India provides the most up-to-date information.
Italy, Portugal, Slovakia, and Bahrain are the only countries that provide a glossary of relevant terms in their OGD portals, and only 11 portals allow external users to add new data. With respect to the use and reuse of the data provided through these OGD portals, 23 countries are developing applications for the web and for mobile phones/tablets, in the view that these are a practical and useful means of supplying PSI to general users. The country with the most such applications on its OGD portal is the United States, with 342, well above the average, of 42 applications per country.
In line with the information classification scheme set out in PSI Directive 2003/98/EC (EC, 2003), most of the catalogues examined in the OGD portals contain information of a social nature (44%), while meteorological content is the least commonly found, with 2%. Regarding the number of catalogues, the portals have an average of 18 catalogues. The total volume of data is the dimension that presents the largest degree of dispersion. Thus, China and United States provide the most data volume, with 97,383 and 90,641, respectively, well above the average of 12,975. Regarding the number of agencies that provide data, France and New Zealand host the largest volumes of data per agency.
Finally, we analyzed the types of filters that can be applied in data search. The most commonly used approach was to filter data by categories. This method is possible in 75% of the OGD portals, conversely, filtering by links is the least commonly found, in only 8% of OGD portals. The country that uses most filters for data search is Australia.
Explanatory Analysis
Our explanatory analysis began with an examination of the relationships between the factors that affect the quality of the OGD portals. Table 4 shows the Pearson correlation matrix obtained, which reflects two medium-strength correlations between the variables “average degree of Internet penetration” and “economic capacity” and between “corruption” and “online administration,” together with other weaker correlations of scant importance. We detected no problems of multicollinearity that might compromise the model.
Pearson Correlation Matrix.
*Correlation is significant at the 0.10 level (2-tailed). **Correlation is significant at the 0.05 level (2-tailed). ***Correlation is significant at the 0.01 level (2-tailed).
Table 5 shows the Tobit regression results. The QI model is statistically significant, thus confirming the appropriateness of applying Tobit regression analysis to dependent variables of this type.
Results of the Tobit Regression Analysis.
Note. LR = logistic regression; QI = quality index.
*Significant at the .10 level (two tailed). **Significant at the .05 level (two-tailed). ***Significant at the .01 level (two tailed).
Regarding the significance of the explanatory factors, the Tobit regression analysis shows that both economic capacity and Internet penetration are significant variables (see Figure 1). It is important to note that the economic factor (i.e., economic capacity), measured in terms of GDP, is a key variable in determining the quality of OGD portals. These results are in line with those obtained by Gandía and Archidona (2008) and De Andrés et al. (2010), in that it is the countries with a greater economic capacity which are at the forefront of promoting higher quality OGD portals.
A positive relationship was observed between Internet penetration and OGD portal quality, which confirmed the expected relationship. Thus, countries with a higher level of Internet penetration will tend to present higher quality OGD portals. These results are consistent with those obtained by Gandía and Archidona (2008) who concluded that when there is a large population with Internet access, the government is more likely to consider its OGD portals a useful instrument for information disclosure and the implementation of e-government. And the more information that is disclosed, the higher the quality of the OGD portal.
Conclusions
In the field of public administration, OGD portals have revolutionized the concept of online access to data, given their usefulness and the possibilities for reuse of data by different stakeholders. However, despite the benefits offered by OGD portals for governments and for society in general, many developed and developing countries have yet to take advantage of the economic, social, and governance potential offered by these portals. Moreover, despite their increasing prevalence, to date little research attention has been paid to the question of OGD portal quality.
As is made apparent throughout this study, if an OGD portal is to be developed to its maximum potential and to the highest possible level of quality, public data should be supplied in a standard form, so that it can subsequently be reused. However, our analysis reveals considerable heterogeneity among the OGD portals analyzed. There exists a wide range of structures, with different processes for data search, different formats for presenting information, and a diversity of means to classify the information. In short, there is no single model. In response to this situation, work should be done to standardize PSI and coordinate access to it, in accordance with information quality models set out in standards and international agreements. The existence of such standards would be of great use in facilitating the organization, provision, and evaluation of information via these portals.
Furthermore, governments should encourage active public participation in the use of OGD portals, inviting citizens to collaborate in the provision of new information and the reuse of existing data. At present, however, most countries seem to be making little use of the feedback possibilities offered by their portals, and in many cases do not allow the introduction of new data. It would be desirable to implement a system for measuring the popularity of their data sets in order to identify and clarify the information needs of society and thus enable more such data to be published.
The results obtained regarding the explanatory factors show that OGD portal quality can be affected by the economic costs that are necessarily incurred in achieving acceptable levels of quality, and so a country’s economic capacity is a determinant factor in its ability to achieve a high level of OGD portal quality. Also significant is the extent to which the OGD initiative is being developed and implemented in different countries; where citizens make greater use of the Internet, the quality of OGD tends to be higher, either due to pressure from the users themselves or because governments seek to meet their own requirements.
Looking ahead to possible areas for future research, it would be very useful to extend the explanatory potential of the research by studying other determinants of OGD portals quality, such as examining the impact of human capital and of the government funds provided for the purpose of enhancing OGD quality. Another interesting aspect would be to observe the evolution of the OGD quality in the countries analyzed in the present study and to evaluate the policies adopted to promote PSI services.
Finally, in view of the scant previous research into the local or regional provision of OGD, a study should be performed of these more proximal levels of government, to determine the impact of political and economic factors on their development of OGD portals, and on the OGD quality achieved. In this sense, this article could constitute the base framework for the nonnational studies still underexplored, by taking advantage of the metrics used to measure the quality of OGD portals.
Footnotes
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was carried out with financial support from the Regional Government of Andalusia (Spain), Department of Innovation, Science and Enterprise (Research Projects P10-SEJ-06628 and P11-SEJ-7700).