How to create and use a national cross-domain ontology and data infrastructure on the Semantic Web

1. Extending the layer cake model

The Semantic Web (SW) sees the Web as an interlinked collection of data (Web of Data) instead of only a space of interlinked hypertext documents, Web of Pages. The idea was proposed in the 90’s by Tim Berners-Lee [4], and first recommendations for the SW,1

To develop applications, the layer cake model is not enough: domain and application specific infrastructures based on shared W3C standards and best practices are needed, too. These can focus on specific domains, such as medicine, biology, cultural heritage, or geography on an international level. However, in practice one also has to deal with national level issues and data available that are represented using national languages, data models, vocabularies, and are created using conventions of local legacy systems. It therefore makes sense to speak about a “national Semantic Web infrastructure” that is compatible with the global SW standards. For example, Cultural Heritage (CH) data in different countries is often nationally specific calling for adapted local solutions for representing and using the data.

Most of the international infrastructure work [5] is focused on collaborations on particular application domains. In contrast, this paper concerns the question: How to Create a National Cross-domain Ontology and Linked Data Infrastructure and Use It on the Semantic Web. This problem is addressed by presenting, discussing, and evaluating approaches and living laboratory experiments developed in Finland during 2002–2023. Presenting lessons learned in this particular endeavour is hopefully useful in a more general setting, as similar challenges are likely to be faced in other countries, too.

The paper is organized as follows: In Section 2, elements needed for a national SW infrastructure are first introduced. The idea and lessons learned in developing a national ontology and a LOD infrastructure are then presented in Sections 3 and 4, respectively. After this, applications of the infrastructures are discussed: as a proof-of-concept, a model is presented that has been used for creating a series of in-use data services and semantic portals that have had up to millions of users (Section 5). Finally, contributions of the work are summarized and related works discussed (Section 6). This paper presents the first consolidated account of this line of research and development, summarizing works reported before in over 500 papers and other publications available on the Web.3

OPEN IN VIEWER

Fig. 1.

Elements needed for a national Semantic Web infrastructure.

Figure 1 depicts components that are arguable needed in the developing a national SW infrastructure – according to the experiences to be reported in this paper. The system is based on the domain agnostic W3C Web Standards and Best Practices (on the left below in the figure) of publishing Linked Data4

For developing, maintaining, and using the infrastructure in a sustainable way a Human Infrastructure is needed (on the left in Fig. 1), too. This involves, e.g., educating people about the technology, introducing SW courses in university curricula, and production of documentations and learning materials for the community using national languages. In the Finnish case, for example, online materials have been created,8

In the early 2000’s, the focus in SW research was on ontologies [76], arguably the “silver bullet” of the SW [10]. Accordingly, a series of projects called “FinnONTO” in 2003–2012 were conducted12

A National Effort The goal in the FinnONTO initiative [45] was to develop a national ontology and content infrastructure, based on W3C standards, that would be cross-domain,13

The initiative produced 1) metadata models contributing to national standards,14

A central goal of FinnONTO was to create an interlinked cloud of national ontologies [11] based on existing thesauri that were already used in different areas of the society. The rationale for this was that metadata available in national databases had already been catalogued using these thesauri, which would make it much easier to develop applications. According to the FinnONTO vision, the ontologies should be served not only through human readable browser interfaces,15

From Thesauri to Ontologies The FinnONTO project transformed 16 key national thesauri used in Finland into light-weight ontologies listed in Table 1. The transformation process was more ambitious than just transforming the traditional standard thesaurus format [2] into an RDF-based model, such as SKOS.16

Linked Ontology Cloud KOKO The domain ontologies in Table 1 contain similar and related concepts [11]. Especially, the largest ontology YSO ( 27 200 concepts) – transformed from the most used thesaurus YSA of the National Library in Finland – shared lots of concepts with all other ontologies, in some cases more than 50%. This suggested that the ontologies should be linked together using YSO as the top ontology. This idea resulted in creating the Finnish linked ontology cloud called KOKO18

While transforming the top ontology YSO from a thesaurus into an ontology its concept labels were also translated into English and the existing partial Swedish translation was completed. This makes it possible to align the ontology with related international domain ontologies. In some other KOKO ontologies translations were already available.

Lessons Learned A key problem to be solved in FinnONTO was that large cross-domain thesauri, especially the General Finnish Thesaurus YSA, could not anymore be maintained easily by its management team. Even if the team included people from different fields, the terminology related to specific areas needed deeper domain specific expertise than was available. Developing the interlinked KOKO ontology cloud mitigates the problem by distributing work on specific concepts to collaborative, domain specific ontology developer teams. However, in this model new problems arise pertaining to maintaining the linked ontology cloud and to coordinating the collaboration network [11]. These new challenges are now being tackled by the Finto collaboration network19

A challenge encountered in the ontologization process was that organizing the concepts into class hierarchies cannot in many cases represent correctly the meaning of the original terms that can be complex and fuzzy. The world cannot be represented fully using ontologies and there can be several ways in which this can be done. In spite of such challenges, the idea of adding more semantics needed for application development seems to be a better option than continuing using the original thesauri, whose semantics were too vague from a sofware development point of view in applications, such the semantic portals to be discussed in this paper. A strategic choice made in FinnONTO was to follow the wisdom articulated by Jim Hendler already in the late 90’s in the SHOE project:20

A mundane challenge of developing large domain ontologies, at least in Finland, is how to convince the funding organizations, year after year, that this never-ending work should be supported on a regular basis, not only as separate short-time projects. This is possible if the benefits of using ontologies in practical applications can be demonstrated. In our case, it took some ten years of project-based work before the KOKO ontology infrastructure and the current Finto.fi services could be funded in a more sustainable way by two Finnish ministries. The strategy taken in FinnONTO was to move forward in baby steps, and after each step show a demonstrator on how the ontologies can be applied in practise for creating something useful.

The idea of creating a “living laboratory” of ONKI ontology services [85,91] on the Web turned out to be important for deploying the infrastructure. The participating FinnONTO organizations were supported by the project in connecting their legacy systems to the APIs of ONKI for testing and evaluating the services. Finally, the “point of no return” was reached where pulling off the plug of the services was not an option anymore as the number of ONKI API users were counted already in hundreds.

The FinnONTO project series 2003–2012 started with a smallish one-year project, but eventually grew into a national effort of substantial size on the Finnish scale with dozens of funding organizations involved. A reason for this was that in addition to public organizations, such as museums, libraries, and archives, also companies got interested in the technology, which convinced the main funding organization Tekes (called today Business Finland) that something useful and of monetary value is happening related to semantic web technologies. It is usually easier to get funding for technology development than for research in humanities.

The KOKO ontologies are based on keyword thesauri whose terms usually correspond to the classes. FinnONTO worked also on various “instance-based” ontologies, such as national geogazetteers, person and organization registries, biological taxonomies of species [87,88], and nomenclatures and terminologies of medicine [78], such as Medical Subject Headings MESH.21

When developing ontology-based applications in FinnONTO, much of the time of the developers was “waisted” in cleaning and aligning the data from different organizations for interoperability. Obviously, it would be more cost-efficient do this work already when cataloging the data using ontology services. This would also enhance the quality of the linked data, which is a critical problem [93] on the SW. The local cataloguers know best their own data and should have the best interest in quality of their data. The motto for the FinnONTO work was therefore taken from a wisdom of Albert Einstein: Intellectuals solve problems – geniuses prevent them; a key goal of FinnONTO was to prevent interoperability problems rather than to solve them afterwards when the damage has already been done in cataloguing [21].

A major outcome of FinnONTO was the ONKI ontology server with its ontologies [91] that were published first in 2008. As a next step, the ONKI Light service26

The Finto.fi service has grown into a popular national open service. In 2019 it was used by 280 000 different users and its APIs were called 32 million times. The users include, e.g., museums, whose cataloging system get their keywords with URI identifiers from Finto. These developments suggest that the fundamental ideas of FinnONTO are feasible; they have actually made a paradigm change in Finland in developing and using linked light-weight ontologies on a national level instead of thesauri.

The SW infrastructure model of Fig. 1 includes a platform for publishing datasets and (re-)using them via web services. A key component in LD publishing is the SPRAQL endpoint, but the platform should also support other functions [16]. The Linked Data Finland service LDF.fi30

LDF.fi has two user-groups: 1) For application developers, LDF.fi provides SPARQL endpoints and a suite of standard Linked Data (LD) services, including content negotiation, APIs for downloading datasets, LD browsing and editing, and additional tools for, e.g., data documentation and visualization. 2) For data publishers, the idea is to support and automate the data publishing process in the following way: The publisher creates a service description of the dataset and its schemas, using an extended version of the W3C Service Description recommendation.32

Linked data publications on the SW are typically evaluated with the W3C “5-star” deployment scheme,34

Schemas can be documented automatically in LDF.fi for the human reader using a schema documentation generator, in our case using SpecGen38

LDF.fi is implemented by a combination of the Fuseki SPARQL server41

Lessons learned The Linked Data Finland platform has turned out to be useful for data-analytic research purposes and in developing applications (cf. Section 5). LDf.fi has been used for publishing some 100 linked datasets. Many of them are in use in semantic portal applications and via SPARQL querying combined with query editing and scripting tools using the open CC BY 4.0 license. Some datasets are used only internally in related research projects, and for some datasets licensing policy of the data owners prohibits open use. LDF.fi hosts several instance-based ontologies, too, such as an RDF-based version of the ca. 800 000 official Finnish geographical places based on data of the National Survey.

The LDF.fi service is still maintained by the Aalto University and University of Helsinki that developed it on an academic project basis, but with the hope that some day it will be deployed and be maintained in a more sustainable way – this is at least what happened to the related ONKI/Finto ontology services. A step towards this is that in 2020 the idea of providing Linked Open Data services on a national level using LDF.fi was accepted on the new research infrastructure roadmap of the Academy of Finland as part of the larger FIN-CLARIAH infrastructure.

When developing the Finnish SW infrastructure, applications that test and demonstrate its usability were constantly developed. This work evolved gradually into a set of principles for developing LOD services and semantic portals on top of them, called the Sampo Model, and the Sampo Series of LOD services and semantic portals [27]. The novelty of the Sampo model44

The Sampo Model is an informal collection of principles for LOD publishing and designing semantic portals listed in Table 2. Principles P1–P3 can be seen as a foundation for developing data services; P4–P6 are related to creating semantic portals. The model is based on the idea of collaborative content creation (P1). The data is aggregated from local data silos into a global service, based on a shared ontology and publishing infrastructure (P2). The local data are harmonized and enriched with each other by linking and reasoning. In this model everybody can arguably win, including the data publishers by enriched data and shared publishing infra, and the end users by richer global content and services. The model argues for the idea of separating the underlying Linked Data service completely from the user interface via a SPARQL API (P3). This arguable simplifies the portal architecture and the data service can be opened for data analysis research. For example, YASGUI45

The general idea of principles P4–P6 is to “standardize” the UI logic so that the portals are easier to use for the end users and for the programmers to develop, as demonstrated in the Sampo-UI framework tool [46,70]. Principle P4 articulates the idea of providing different thematic application perspectives by re-using the data service. The application perspectives can be provided on the landing page of the Sampo portal system or be completely separate applications by third parties. According to P5 the application perspectives can be used by a two-step cycle for research: First the focus of interest, the target group, is filtered out using faceted semantic search [39,84,90]. Second, the target group is visualized or analyzed by using ready-to-use data analytic tools of the application perspectives. Finally, the Sampo model aims not only at data publishing with search and data exploration [63] but also to data analysis and knowledge discovery with seamlessly integrated tooling for finding, analysing, and even solving research problems in interactive ways (P6) [25].

The Sampo model principles are compatible with the FAIR principles for creating Findable, Accessible, Interoperable, and Re-usable data,48

The Sampo model has evolved gradually in 2002–2023 via lessons learned in developing a series of semantic portals and LOD services, starting from MuseumFinland – Finnish Museums on the Semantic Web49

Developing these systems in use in a university research group64

Data about all over 20 Sampo portals, including links, videos, publications, and further information are available on the Sampo portals homepage.65

This paper addressed challenges of extending the SW layer cake model for creating ontology and LOD infrastructures on national and domain specific levels. Lessons learned in developing Finnish ontology and LOD services 2002–2023 for practical applications were discussed. This work has utilized methods of design science [17,62,68] and action research [8], where the idea is to design artifacts, evaluate their value and utility, and to provide improvements in solutions. Rather than creating theoretical knowledge, design science applies knowledge. In this paper, infrastructure elements were designed, implemented, and applied to create the Sampo series of data services and portals as proof-of-concepts. They have had up to millions of end users, which suggests feasibility of the national infrastructures presented. The line of R&D presented is unique in its focus on different domains on a national level, longevity, and series of applications in use on the Semantic Web. Our general strategy has been to develop useful proof-of-concept prototypes and to publish them openly on the Web for everyone to use. The data owners and stake holders, such as memory organizations, saw this as an opportunity to develop their own systems, started to use the services and applications, and in many cases the point of no return has been reached.

In contrast to current related ontology library systems [7,66] that typically focus on particular application domains, ONKI and Finto aimed at being a cross-domain ontology service on a national level. For example, the BioPortal [72] of Stanford University is focused on publishing biomedical ontologies. There are lots of LOD services and SPARQL endpoints around.66

OPEN IN VIEWER

Fig. 2.

Timeline illustrating the development of the Semantic Web and work reported in this paper.

Figure 2 illustrates the work reported in this paper on a timeline and in relation to some global developments of the Semantic Web. The development of the SW was boosted by the seminal article in Scientific American in May 2011 by Tim Berners-Lee, James Hendler, and Ora Lassila [3], and this inspired us to organize the conference Semantic Web Kick-off in Finland in the autumn of 2001 together with researchers of the W3C, the Finnish Artificial Intelligence Society, and some other organizations [20]. This event initiated SW research in Finland. During the past 20 years, the SW has evolved in phases [18] (cf. top of the figure) with a focus first on ontologies [76], then on Linked (Open) Data [16], and today on Knowledge Graphs (KG) [15]. In the same vein, our work on infrastructure first focused on ontologies and ontology services (ONKI.fi and Finto.fi) and then on Linked Data and data model services. The Sampo series of applications reflects this development by showing a shift of focus in three generations [25] from data publishing, based on shared ontologies and metadata vocabularies67

The ontologies and linked open datasets discussed in this paper constitute together a kind of national Linked Open Data Cloud. The idea is that as new ontologies and applications with new datasets are developed, the open RDF data already available in the infrastructure, say ontologies of places and historical people, can be reused and refined gradually better and better. This applies also to the open source tools, such as the Sampo-UI framework, that has been re-used and extended in all Sampo systems after publishing NameSampo in 2008. A goal of the current national FIN-CLARIAH infrastructure initiative is to foster this development.

The experiences reported in this paper indicate that creating and using a national semantic web infrastructure is useful from the data producers’ and data users’ points of view. However, creating and using linked data has its own challenges, too. More collaboration and agreements on data models and ontologies are needed for interoperability between the data producers, which complicates the publication process. Integration of SW technologies with legacy systems may be challenging, and there is lack of IT personnel competent in using SW technologies and tools. Creating linked data manually is costly but automatic methods may not be available and automation lowers data quality. Using structured semantic data and making the knowledge structures explicit to the end user in the UI calls for new kind of digital data literacy and source criticism68