Abstract
Since 2014, the evaluation system for government-funded research institutions (G-FRIs) has undergone significant reforms aiming to improve the quality and utilisation of performance outcomes. The performance evaluation system for national R&D projects has also been enhanced to encourage the active dissemination of research outputs.
The present study aims to analyse the relationship between the changes in the performance evaluation system and the main research and development outcomes of these institutions, while considering the period before and after these systemic improvements. This study analyses whether the improvement of the evaluation system and performance indicators over the past decade (2011–2020) has had a significant impact on scientific and technological achievements, economic performance, and diffusion of achievements.
The research results revealed that there was a decrease in overall scientific and technological outcomes, particularly in the number of patents granted, following the improvements made to the performance evaluation system. However, it was also observed that certain aspects of performance dissemination showed significant positive effects as a result of the evaluation system improvement. Specifically, the analysis indicated that there were significant impacts on technology transfer rates relative to Article publication and patents, suggesting that the evaluation system improvement had positive effects in some aspects of performance dissemination indicators.
Keywords
Introduction
Institutional evaluations of government-funded research institutes (G-FRIs) have evolved significantly since the science and technology research council system was first established in 1999. This evolution can be categorised into two distinct periods: Up to 2014, the evaluation followed a common standard approach, with all G-FRIs assessed annually (from January to April) in terms of their management performance (evaluated yearly) and, in some years, in terms of their research project performance (evaluated every three years). During the time when institutional evaluations of G-FRIs were assessed using a common standard, the research sector was evaluated based on quantitative indicators of primary research outputs, such as Article publication, patents, and technology transfers.
Since 2014 in particular, the research project sector has been encouraged to improve the quality of individual projects and tasks by setting autonomous performance goals and indicators that reflect the unique characteristics of each institute. In the management sector, the evaluation system has been developed by diversifying the indicators used for performance utilisation and dissemination and assigning them higher weights.
Meanwhile, the performance evaluation of national R&D projects (standard performance indicators, 2014–2020) shifted its policy focus from the quantitative expansion of scientific and technological outputs to the utilisation of performance indicators as key evaluation elements. The main goal was to promote the dissemination of research outcomes (conversion of articles into patents, utilisation of articles in technology transfers) and to enhance economic outcomes (technology transfers and royalties). This approach focuses on the economic and social utilisation, dissemination, and impact of research and development outcomes. Furthermore, National R&D funding programmes guided not only by direct outcomes, but also by performance impact indicators such as commercialisation success through technology transfer, corporate revenue growth, employment effects, and import substitution This study raises the need for research under the assumption that significant changes in major evaluation policies and directions, such as institutional evaluations and national R&D project evaluations, have influenced the key research outcomes and growth of the evaluated G-FRIs.
This study aims to assess the practical effectiveness of evaluation policies by analysing key R&D performance outcomes of G-FRIs before and after the improvement of performance evaluation policies. The scope and direction of this research include 24 institutions under the jurisdiction of the National Research Council of Science and Technology (NST); there are 25 such institutions in total. This study covers G-FRIs, accounting for 50% of the total. The remaining 25 institutes, managed under separate evaluation systems, were excluded from the analysis.
This study intends to analyse the institutions’ investment resources, such as their project funds and personnel, in relation to their outputs, like Article publication, patents, technology transfers, royalties, and a compound index of performance diffusion that includes the transfer of technology resulting from the institution’s articles and patents.
The aims and procedure of this study are, first, to analyse whether changes in a nation’s science and technology evaluation policy have had a tangible impact on research groups such as G-FRIs and the performance of national R&D programmes. Second, based on the analysis results, it assesses which performance indicators were affected by the policy changes and proposes directions for managing Korea’s research institutes and national R&D programmes. Third, it suggests the potential applicability of the findings as reference material for emerging countries that are in the process of establishing their national science and technology policies—particularly in designing R&D evaluation systems, performance indicators, and periodic assessment frameworks.
The rest of this study is structured as follows. Overview of changes in the performance evaluation system related to G-FRI and Review of literature involving performance analyses of G-FRI. Next, presents the research design, including the hypotheses and methods. Finally, summarises the analysis results of the hypotheses and synthesises the results while reviewing the significance and limitations of this study.
Overview of the Evaluation System for G-FRIs in Korea
The subject of this study, ‘G-FRIs in the field of science and technology’, refers to institutions that receive government funding and primarily engage in research in the field of science and technology. Prior to this, let us first examine the current status of GERD (Gross Domestic Expenditure on R&D) in South Korea. Table 1 presents the allocation of GERD funding between the public and private sectors relative to GDP for the period 2011–2023.
Status of Expenditure GERD to GDP, Scale of Public & Private*.
**GERD (gov) represents government-funded R&D expenditures, while official statistics on private sector investment are not available.
***Public GERD: Central Government, Public Institutions, and G-FRIs.
****Private GERD: Universities, Large Enterprises, Medium & Small Enterprises, Others.
Table 2 provides an overview of the current status of the 25 G-FRIs in the field of science and technology under the National Research Council of Science & Technology (NST), which falls under the Ministry of Science and ICT.
Status of G-FRI in the National Research Council of Science & Technology.
Apart from these, there are 25 R&D institutions directly overseen by respective government ministries, which were not included in this study, as they operate under a different evaluation system.
The operation and funding of G-FRIs are primarily supported by government appropriations and other sources of revenue. The majority of funding for most research institutions, constituting 87% of the total institution’s budget at G-FRI, is government funding administered through national research and development (National R&D) projects, as indicated in Table 3. The remaining funding sources include private contracts and self-generated revenue, such as technology fees. The proportion of government funding varies from 15% to 85%, depending on the nature of the specific projects undertaken by each G-FRI.
Financial and Personnel Status of G-FRI in the Field of Science and Technology.
Overview Evaluation System for G-FRIs
G-FRI has implemented performance evaluation systems, as presented in Table 4, that primarily consist of Institutional Evaluation and National R&D Project Evaluation. Institutional Evaluation encompasses an assessment of the overall research and development activities and management of the institution as a whole.
Overview of the Performance Evaluation System Related to G-FRI.
On the other hand, the National R&D Project Evaluation focuses on evaluating the performance of research institutions on a project-by-project basis, with a specific focus on contract-based projects and tasks. The scope includes the institutional evaluations of 24 G-FRIs and approximately 700 national science and technology R&D programmes, such as the Green New Deal climate technology development projects they undertake.
The evaluation system for G-FRI under the auspices of the science and technology sector underwent significant changes in four phases: First, during the initial period of the reorganisation of G-FRI under the scientific research council (1999~2007), an annual quantitative research performance indicator system was operated in a collective manner that encouraged quantitative growth in research outcomes.
During this period, key performance indicators—such as article publications (SCI, KCI), patents (application, registration, domestic & foreign), technology transfer, licensing income, and external collaboration—were quantitatively evaluated for each research institute based on year-over-year improvement and goal attainment. The results were used as a reference for determining government funding allocations. In Table 5, From 2014 onwards, with the establishment of the Integrated National Research Council for Science & Technology, an institution-specific mission-oriented evaluation system was introduced.
Summary of Changes in the Performance Evaluation System Related to G-FRI.
**National R&D Project Performance Indicators (2014~2015), Korea Institute of Science and Technology Evaluation and Planning.
The National R&D Programme Performance Evaluation System has undergone significant changes since the unveiling of the 2nd Performance Evaluation Basic Plan in 2011. The fundamental approach adopted during this period involved a shift from quantity-focused evaluation indicators to quality-focused ones. The period following the official announcement of the 4th standard performance indicators in December 2014 marked a significant moment when these standard performance indicators, particularly those related to qualitative metrics and performance dissemination (commercialisation), started to be extensively emphasised and incorporated.
Review of Literature
Previous studies on perspectives of policies and evaluation systems are as follows.
The theoretical foundations of this study are grounded in several seminal works. Schumpeter (1942) emphasised not only the promotion and accumulation of economic growth, but also the significance of structural transformation driven by innovation, which he argued could be facilitated through institutional intervention by the state. Building upon this premise, Lundvall (1992) formally introduced the concept of the National Innovation System (NIS), theorising that innovation does not stem solely from R&D activities, but rather emerges from a social and institutional process of interactive learning involving actors such as governments, firms, and educational systems. This perspective influenced the view that R&D policy should not focus solely on scale or volume, but rather on the connectivity and institutional arrangements within the innovation ecosystem. Subsequently, Freeman and Soete (1997) further expanded this line of thought by highlighting the importance of social and institutional factors, such as education systems, regulatory environments, and the broader R&D infrastructure. They argued that governments should act not merely as financial supporters but as institutional coordinators that guide and shape innovation environments. Moreover, they asserted that innovation performance cannot be adequately captured by quantitative inputs or outputs alone, such as R&D expenditure, publication counts, or patent numbers. Instead, they advocated for the use of dynamic impact-oriented indicators, including technology commercialisation, socio-economic spillovers, and employment generation.
In the domain of public policy evaluation, Rossi et al. (1979) established that the primary purposes of evaluation in the public sector include assessing the effectiveness and efficiency of policies and programmes, supporting informed decision-making (e.g., budget allocation, programme continuation), and ensuring accountability and public trust.
Jeon et al. (2022) discuss the Composite S&T Innovation Index (COSTII, KISTEP) as a method for assessing science and technology innovation capabilities. This approach identifies critical factors affecting national innovation capacity, such as human resources, R&D investment, and the number of patents (collaboration), which are refined and utilised as analytical indicators.
From a government policy perspective, Yu et al. (2021) actively reviewed the possibility of transforming the current national R&D performance evaluation system from a short-term outcome-oriented evaluation to a dual evaluation system comprising both short-term and medium- to long-term outcomes.
Krishna (2014) examines how the governance systems of science, technology, and innovation (S&T&I) are evolving in response to the transition towards a knowledge-based economy. The study particularly analyses issues such as performance-oriented funding, the strengthening of research evaluation systems, and enhanced institutional accountability in developing and emerging economies. It highlights the shift of traditional national R&D policies toward evaluation-driven and performance-based management frameworks to adapt to the demands of global competition.
Lee and Nam (2020) analysed factors influencing the acceptance of institutional evaluations and found that trust in the government’s evaluation system and the desire to receive favourable evaluations significantly affect acceptance. These findings suggest that changes in evaluation systems may influence both the attitudes and performance of institutional staff seeking better evaluations.
Previous studies on performance outputs relative to resource inputs are as follows.
Marinova et al. (2008) assess the innovation performance of technologies generated by GRIs and universities using metrics such as patent counts, commercialisation rates, corporate R&D investment, and productivity gains. The study positions GRIs as key technology providers and emphasises the need for governments to establish effective governance frameworks to facilitate technology transfer.
In the context of R&D output and productivity, Griliches (1980) estimated the marginal product of additional R&D input, such as budget or personnel, and demonstrated that beyond a certain level of investment, returns tend to decline, indicating the presence of diminishing returns.
Crespi and Geuna (2004) provided an empirical cross-country analysis of the productivity of science, focusing on how scientific outputs vary across nations relative to R&D expenditures and manpower inputs. Their study highlights the need for performance-based R&D systems, emphasising the importance of evaluating the efficiency and effectiveness of science and technology policy in different national contexts.
Brown and Svenson (1998) argued that R&D research organisations are structured as systems, with manpower and budget (Funds) being key input factors for productivity measurement, and articles, patents, etc., being indicators of output performance. They structurally demonstrated the diverse input and processing output systems of research organisations and emphasised the importance of outcome measurement.
Min and Park (2013) conducted an integrated performance analysis by applying value weights to each of the performance factors, such as research funds, research personnel, and research support personnel, to analyse their impacts on technology fees, Article publication, and patents. They found that research personnel, research support personnel, and research funds were the most influential factors.
Altogether, these studies shed light on the various factors involved in influencing research and development performance in G-FRI and their relative importance.
Research Design
The analysis in this study focuses on 24 G-FRI affiliated with the National Research Council of Science & Technology (NST), as presented in Table 2. One institute (GTCK) was excluded from the analysis due to a lack of available data.
Table 6 shows the data names and their source, Table 7 presents descriptive statistics for G-FRI’s Resources and R&D Performance (2011~2020). The scope of the analysis covers data from 2011 to 2020, spanning ten years, using publicly available or separately obtained information from each G-FRI.
Basic Data Sources.
Summary Statistics for 24 G-FRI by Year, 2011–2020.
As reviewed in the context of policy improvements, the institutional evaluation and R&D project performance evaluation of G-FRI have undergone significant advancements over the past ten years.
First, in institutional evaluation, a transition from short-term quantitative evaluation to medium- to long-term mission-oriented evaluation has taken place since 2014. As a result, research institutions have experienced a reduced burden of having to expand quantitative outcomes, such as Article publication and patents. Moreover, the national R&D project performance evaluation has focused on improving the standard performance indicator guidelines since around 2014, strengthening indicators to promote the utilisation of technological development outcomes and emphasising the improvement of the qualitative level of research outcomes. These major changes in the two evaluation systems began to become distinctly recognisable around 2014–2015, with the first substantial effects of each evaluation system emerging from 2015.
To support the development of hypotheses, previous studies examining how government evaluation policies influence the direction and attitudes of research institutions toward performance management were reviewed. In addition, interviews were conducted with six evaluation managers from NST-affiliated G-FRIs and government-direct research institutes. The G-FRIs under NST mentioned that changes in institutional and national R&D evaluation policies have influenced their performance management policies and individual evaluation indicators. Meanwhile, the government-direct research institutes stated that these changes have had a significant impact on performance management planning and organisational evaluation. Most institutions responded that these factors have affected their performance in technology transfer and royalty income.
The policy direction changes in these two evaluation systems, as summarised in Table 8, emphasise the importance of G-FRI encouraging medium- to long-term qualitative improvements in R&D. Moreover, it highlights the necessity of focusing on the diffusion of key R&D outcomes, such as articles and patents, rather than prioritising short-term quantitative achievements. Such changes in evaluation policies suggest that they may influence trust in and compliance with the government’s evaluation system, as well as the attitudes and performance of members within the evaluated institutions who aim to receive favourable assessments.
Hypothesis Formation.
For the analysis in this study, the forms of outcomes produced by G-FRI are categorised according to the National R&D Programme Standard Performance Index Guidelines, which are presented in the Table 9.
National R&D Development Project’s Standard Performance Indicators.
Considering the characteristics of the previous evaluation system improvements and the classification system for performance indicators, this research aims to establish fundamental analytical perspectives and hypotheses & Sub-Hypotheses as follows:
H1: The scientific and technological outcomes relative to the research resources invested by G-FRI will show a significant difference before and after the improvement of the evaluation system. After policy improvement, there will be either no impact or a decline in performance. H1–1: The performance of the Article publication will either be negatively impacted or have no significant effect following the input of project funds, manpower, and policy improvement. H1–2: The performance of patents will either be negatively impacted or show no significant effect following the input of project funds, manpower, and policy improvement. H2: The economic performance and performance dissemination outcomes of G-FRI will exhibit a significant difference before and after the evaluation system improvement, with improvement trends emerging after the evaluation system improvement. H2–1: The performance of technology transfer and royalties will be significantly positively impacted following the input of project funds, manpower, and policy improvement [Economic Performance]. H2–2: The performance of technology royalties per technology transfer will be significantly positively impacted following the input of project funds, manpower, and policy improvement [Economic Performance]. H2–3: The conversion rate of intellectual property rights (patent performance) relative to articles will be significantly positively impacted following the input of project funds, manpower, and policy improvement [Composite Indicator of Performance Dissemination]. H2–4: The implementation rate of technology transfer relative to Article publication will be significantly positively impacted following the input of project funds, manpower, and policy improvement [Composite Indicator of Performance Dissemination]. H2–5: The implementation rate of technology transfer relative to patents (registrations) will be significantly positively impacted following the input of project funds, manpower, and policy improvement [Composite Indicator of Performance Dissemination].
The prior studies reviewed in relation to this research primarily focused on analysing the relationship between independent variables and key research outcomes based on the type of input resources.
In contrast, this study emphasises analysing the impact of research resources allocated to G-FRIs and improvements in major performance evaluation systems. This approach distinguishes it from the hypotheses and research models discussed in previous studies in this field. In particular, this study regards the improvement of performance evaluation systems as a critical factor and focuses on how changes in the policy environment and factors influencing performance generation have evolved at a specific point in time (2014–2015).
Table 10 Shows the analysis model. The methodology used to verify the hypotheses set in this study involves multiple regression analysis. To analyse the extent of influence between the independent variables, which are input factors such as resource input variables like project expenses and personnel, and the dependent variable, which is the performance output variable, the study also incorporates a key variable of ‘policy improvement’ over the past 10 years. This variable is divided into two periods, pre-policy improvement (2011–2014) and post-policy improvement (2015–2020), which are represented in the form of a dummy variable.
Analysis Model.
Regarding the performance output variables, this study classifies them into groups according to the ‘National R&D Standard Performance Indicator Guidelines’. These groups include ‘Scientific and Technological Performance’, ‘Economic Performance’, and ‘Performance Diffusion’ (Table 11).
Setting of Detailed Variables for Analysis.
**Total personnel (including research and support personnel).
***This is designated as a composite performance diffusion indicator in the ‘National R&D Programme Standard Performance Indicator Guide’.
Analysis Results
The analysis of G-FRI’s scientific and technological performance is as follows.
An analysis was conducted to test the detailed hypotheses related to the first hypothesis of the study, which is ‘G-FRI’s scientific and technological performance will show a significant difference before and after resource allocation and policy improvement, with a potential decline or stagnation after policy improvement’.
Tables 12 and 13 presents the results and Robustness Check for Hypothesis 1. The analysis included 240 observations in total, representing 24 institutions over a period of 10 years. The regression model was statistically significant (p < .001). Moreover, the variance inflation factors (VIF) for the independent variables were all below 5 (e.g., 4.95 for project expenses, 4.89 for personnel, and 1.03 for policy improvement), suggesting that there were no issues related to multicollinearity.
Analysis Results of H1.
Robustness Check of H1.
First, H1–1 regarding the impact of project expenses, personnel, and policy improvement on the overall Article publication performance, the coefficient of determination (R2) was 0.278. The analysis results indicate that personnel (Coef = 0.210, p < .01) had a significant positive effect on Article publication performance. However, policy was not statistically significant (Coef = –50.602, p > .05).
Second, H1–2 concerning the impact of project expenses, personnel, and policy improvement on the overall patent performance, the coefficient of determination (R2) was found to be 0.691. The analysis results indicate that project expenses (Coef = –0.143, p < .01) have a significant negative impact on patent performance. On the other hand, personnel (Coef = 2.175, p < .001) have a significant positive effect, and policy improvement (Coef = –322.587, p < .001) has a significant negative impact on patent performance. Therefore, for H1–2, patent performance is negatively influenced by project expenses and policy improvement, thus indicating a decline in quantitative growth after the allocation of project expenses and policy improvement.
To check the robustness of the regression analysis results, the data were divided into two periods: before the policy improvement (2011–2014) and after the policy improvement (2015–2020).
First, according to the H1–1 ‘Article publication’ analysis results, project budgets were not statistically significant either before or after the policy improvement, whereas personnel demonstrated a positive and statistically significant effect after the policy improvement (Coef = 0.179, p < .05). Second, H1–2 ‘patent’ analysis revealed that project budgets had a significant negative effect both before (Coef = –0.270, p < .05) and after (Coef = –0.123, p < .01) the policy improvement. In contrast, personnel exerted a significant positive effect both before (Coef = 3.014, p < .001) and after (Coef = 1.888, p < .001) the improvement.
In summary and discussion of H1, the results of H1–1 showed that personnel had a positive impact on both publication and patent performance before and after the policy improvement, contrary to the initial expectation. This implies that output productivity relative to personnel input continued to improve, independent of external policy changes. This may be attributed to internal performance evaluation systems within each G-FRI and researchers’ intrinsic motivation for academic achievement. Prior Korean studies also highlight the critical role of personnel in driving publication outcomes.
In H1–2, while personnel input remained positively associated with patent performance, project budget input had a decreasing effect following the policy reform. This may reflect diminishing marginal returns and changes in evaluation criteria. Since patents function as strategic assets tied to technology transfer and require continued investment, an excessive volume driven by funding alone may be inefficient. Additionally, reduced emphasis on patent quantity in evaluations after the policy shift likely weakened incentives for numerical growth.
The analysis results for G-FRI’s economic performance and performance dissemination are as follows.
For the second hypothesis, which states that ‘There will be a significant difference in economic performance and performance dissemination of G-FRI before and after resource allocation and policy improvement, and performance will improve after the policy improvement’, the analysis results are as follows. Table 14 presents the results for Hypothesis 2. The number of observations for each variable in the analysis was 240 (24 institutions over 10 years), and the regression model was statistically significant (p < .001), The multicollinearity (VIF) for each independent variable in the analysis was below 5 in every case (budget: 4.95, personnel: 4.89, After policy improvement: 1.03), confirming that there were no issues with multicollinearity.
Analysis Results of H2.
First, H2–1 regarding the impact of project funds, manpower, and Policy improvement on technology transfer and technology royalty performance, the explanatory power of independent variables for technology transfer was found to be 0.443. Manpower (Coef = 0.164, p < .001) had a significant positive (+) impact on technology transfer performance. Meanwhile, the explanatory power of independent variables for technology royalty performance was found to be 0.644. Project funds (Coef = –1.912, p < .001) had a significant negative (-) impact on technology royalty performance, while manpower (Coef = 20.444, p < .001) had a significant positive (+) impact. Policy did not reach statistical significance (Coef = –1,224.428, p = .059).
Second, H2–2 regarding the impact of project funds, manpower, and policy improvement on technology royalties per technology transfer, the explanatory power of the independent variables was found to be 0.110, and none of the independent variables had a statistically significant impact.
Third, H2–3 regarding the impact of project funds, manpower, and policy improvement on patent conversion rate relative to articles, the explanatory power of the independent variables was found to be 0.492. Project funds (Coef = –0.037, p < .001) and policy improvement (Coef = –36.898, p < .05) showed a significant negative (-) impact. On the other hand, manpower (Coef = 0.366, p < .001) had a significant positive (+) impact. Project. funds and policy improvement were found to have a significant negative (-) impact on patent conversion performance relative to the article.
Fourth, H2–4 regarding the impact of project funds, manpower, and policy improvement on the technology transfer implementation rate relative to articles, the explanatory power of the independent variables was found to be 0.152. Manpower (Coef = 0.032, p < .001) and policy improvement (Coef = 7.937, p < .05) each showed a significant positive (+) impact, while project funds did not have a statistically significant impact.
Fifth, H2–5 regarding the impact of project funds, manpower, and policy improvement on the technology transfer implementation rate relative to patents (registered), the explanatory power of the independent variables was found to be 0.134. Policy improvement (Coef = 24.050, p < .001) had a highly significant positive (+) impact, while project funds and manpower did not show statistically significant impacts.
Table 15 presents Robustness Check for Hypothesis 2. The robustness check results for H2–1, ‘technology transfer’, show that project budgets had a slightly positive and statistically significant effect before the policy improvement (Coef = 0.036, p < .05), but no significant effect after the improvement. In contrast, personnel exhibited a consistently positive and statistically significant effect after the policy improvement (Coef = 0.171, p < .001).
Robustness Check of H2.
Meanwhile, the analysis of ‘technology Royalties’ revealed that project budgets had no significant effect before the policy improvement but showed a significant negative effect after the improvement (Coef = –1.981, p < .001). Personnel demonstrated a significant positive effect both before (Coef = 18.072, p < .001) and after (Coef = 21.179, p < .001) the policy improvement.
The robustness check results for H2–2, Personnel, exhibited a slightly positive and statistically significant effect after the policy improvement (Coef = 0.025, p < .05). This result suggests that the productivity of royalties per technology transfer may differ statistically from total royalties.
For H2–3, regarding the ‘Patent conversion per Article’ project budgets slightly negative and statistically significant effect after the improvement (Coef = –0.036, p < .01). In contrast, personnel had a consistently positive and statistically significant effect both before (Coef = 0.343, p < .001) and after (Coef = 0.381, p < .001) the policy improvement.
The H2–4, concerning ‘technology transfer relative to ‘Article publication’ that personnel showed a positive and statistically significant effect after the policy improvement (Coef = 0.038, p < .01).
Similarly, for H2–5, regarding ‘technology transfer relative to patent registrations’, personnel exhibited a slightly positive and statistically significant effect after the policy improvement (Coef = 0.012, p < .05).
In summary and discussion of H2, the results of H2–1 show that personnel positively influenced both technology transfer and licensing income, while the policy improvement had no significant effect. This suggests that technology transfer performance has steadily improved due to internal evaluation systems, income-based incentives, and the accumulation of quality intellectual property.
Since 2011, the government has introduced licensing-based reward systems and reinforced related policies, such as the Technology Transfer Promotion Act (2010–2024). These results imply that, beyond evaluation reforms, broader policy measures have contributed to performance improvements. However, no significant effect was found for the ratio of licensing income to technology transfer, indicating unchanged productivity per transfer unit, which diverges from detailed expectations.
For composite performance dissemination indicators, the results differed from earlier patterns. H2–3 found that personnel positively affected the conversion of articles into patents, aligning with H1–1 and suggesting effective linkage between scientific and technological outputs.
In H2–4, personnel also had a significant effect on technology transfer relative to articles which remained after the policy change. H2–5 showed that the impact of personnel on technology transfer relative to patents became greater post-policy improvement.
Overall, these findings confirm that personnel consistently influenced economic and dissemination outcomes, and that policy reforms have helped strengthen the connection between research outputs and technology transfer. This demonstrates the effectiveness of utilising composite dissemination indicators in national R&D evaluations. Regarding the composite performance indicators related to performance diffusion, both the Article publication-to-technology transfer rate and the patent-to-technology transfer rate showed relatively higher impact levels after the institutional improvements.
This effect is reflected in the fact that technology royalties from G-FRIs surpassed KRW 100 billion annually since 2019. ETRI has led the way with over KRW 30 billion per year from mobile communication and digital technologies. KRICT earned around KRW 4 billion from carbon reduction technologies, and KIST reported similar revenues in biotechnology and advanced materials. These outcomes highlight active commercialisation efforts led by major institutes.
Conclusion
In conclusion, when comparing the detailed performance indicators of G-FRI before and after the institutional improvements, these results support the hypothesis of a decline in patent quantity after the reforms in 2014–2015. However, in terms of performance diffusion, the finding of the positive impacts of personnel on the Article publication-to-technology transfer rate and the patent-to-technology transfer rate suggests that policy improvements partially contributed to enhancing performance diffusion. The implications of this study are as follows.
The primary significance of this study lies in examining how institutional evaluations and improvements in national R&D evaluation policies have influenced the R&D performance of G-FRIs. While previous studies have mainly focused on analysing the effects of input resources, this study assumes and investigates a causal relationship between changes in evaluation systems and shifts in the types of major R&D outcomes.
Given the growing emphasis on intellectual property utilisation within G-FRIs, evaluation systems should be designed to promote the early application of research results. These policies must reflect the scale and characteristics of both institutions and R&D projects to encourage performance management and utilisation from the early stages.
Moreover, since G-FRIs operate under systems that allow for the autonomous setting of goals and indicators in institutional and national R&D evaluations, it is necessary to pursue both continuous policy development and impact assessment to further facilitate the diffusion and utilisation of technological achievements. Based on this, the following policy recommendations are proposed:
First, as demonstrated in this study, it is important for the government to monitor changes in R&D performance around major policy reforms and reflect the findings in future policy enhancements. The pre–post comparison and annual productivity trend analysis used here represent scalable approaches.
Second, within public research institutions such as G-FRIs, it is essential to strengthen the evaluation of technology transfer indicators and promote high-value R&D outcomes that yield greater licensing revenues. To this end, policy frameworks such as the Technology Transfer Promotion Act and the National R&D Innovation Act should be effectively integrated.
Third, the government’s composite indicators for performance dissemination should be actively applied in evaluating national R&D projects. Although these are currently provided as optional indicators at the researcher or project level, their application should be differentiated based on project characteristics.
Fourth, major science and technology management bodies in Asian countries following Korea’s policy path could benefit from incorporating the perspectives and findings presented in this study. In particular, emerging economies such as India, Vietnam, Thailand, Indonesia, and Malaysia may enhance national competitiveness by developing sophisticated innovation outcome indicators that link scientific achievements to economic impact and promote technology commercialisation. R&D indicators and evaluation systems should be adapted to each country’s stage of scientific development, social conditions, and policy objectives.
Finally, since Korea’s national R&D performance indicator system is relatively well established, it is possible to build an effective feedback mechanism that can contribute to the early stabilisation of science and technology policy.
Limitations
This study has limitations in analysing additional outcomes beyond research and development by assigning various types of indicators based on the capabilities and functions of G-FRIs in mission-centric evaluations. Examples include indicators such as ‘performance management, utilisation, and dissemination systems’, ‘support for startups and small and medium-sized enterprises’, ‘external collaboration and openness’, and ‘external communication and information disclosure’.
Focusing on quantitative performance indicators from the national R&D standard performance indicators, this study is limited in its ability to analyse growth or quality through qualitative assessment of articles or patents. Examples of qualitative indicators include academic journals’ ‘standardised impact factor’, ‘citation of articles’, ‘institutional knowledge diffusion’, ‘impact factor’, ‘foreign patents (US, Europe, Japan)’, ‘qualitative evaluations (Korean IP Office)’, and ‘standard patents’.
Furthermore, there may be limitations in focusing on external evaluation systems, such as institutional evaluations and national R&D project evaluations, as the background for changes in the R&D performance of G-FRIs. This is because, for researchers, the structure of organisational and individual evaluations within the institution may serve as a factor influencing performance outcomes.
Lastly, the scope of our data analysis was restricted to 2020 to examine the impact before and after the policy improvements, as the evaluation policies implemented since 2014 have continued without significant changes up to 2024. Nevertheless, it would be meaningful to conduct a follow-up study that supplements the research model using data covering the ten years following the policy improvement.
Footnotes
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.
Funding
The authors disclosed receipt of the following financial support for the research, authorship and/or publication of this article: This research was conducted with support from the Ministry of Trade, Industry and Energy (Korea Institute for Advancement of Technology) under the Convergence Technology Commercialisation and Dissemination Specialist Training Programme (P0014830). Also, this work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korean government (MSIT) (No 2019R1A2C1090655).