The mining of China's policies against COVID-19 based on a policy target and tool co-evolution framework

Abstract

Mining all COVID-19 policies issued by China can provide valuable lessons for both China and other countries in future pandemic control efforts. In this paper, we introduce a novel framework for mining the co-evolution of policy targets and policy tools. We employ bibliometric methods, text mining, and network analysis to explore the entire evolution of China's COVID-19 policies. Examining 1154 central government policies, (a) we extract policy targets from each policy, uncovering their evolution across different stages of the pandemic; (b) propose to identify the policy tool used in each policy unit by integrating an automatic identification model and active learning. We also reveal the categorical structure of these tools; (c) characterize the co-evolution pattern between policy targets and policy tools, shedding light on their dynamic relationship. Our findings indicate that policy targets have shifted across various stages, revealing unique characteristics in China's COVID-19 prevention and control efforts. Notably, there is a self-paradox between prevention measures and economic development. We identify the inadequateness in the distribution and utilization of policy tools. Ensuring the alignment of policy targets with appropriate tools is crucial. This synchronization and co-evolution between policy targets and tools are essential for enhancing the functional approach to policy implementation. This paper is the first systematic mining and review about the COVID-19 policies issued by the Chinese government, and our policy target and tool co-evolution mining framework provides a new quantitative framework for policy mining, especially an improved large language model and active learning theory are integrated to identify the policy tools automatically.

Keywords

epidemic prevention and control policy document mining policy target policy tools policy analysis COVID-19

Introduction

Arguably, one positive outcome of COVID-19 is that it awakened the world to the need for better preparation against future epidemics. Since the emergence of COVID-19 in December 2019, its profound impact on the global stage has been undeniable. However, the consequences have been far from uniform, varying significantly among countries due to the implementation of diverse policies aimed at managing virus mutations, navigating economic challenges, and mitigating the impact on the general population (Dewi et al., 2020). Several nations have implemented contrasting approaches to combat the transmission of COVID-19, with some opting for stringent measures while others pursued a strategy of herd immunity. As consequence, the outcomes in terms of fatalities and economic repercussions have differed significantly (Gupta et al., 2022). While no country can claim to have achieved perfection in their response, it is imperative to thoroughly scrutinize and evaluate these policies to glean valuable insights for future governance during similar calamities.

To stop the next pandemic, we need to draw lessons from our epidemic prevention and control (EPC) policies (Relman, 2020). As a developing nation with a staggering population exceeding 1.4 billion, China has confronted formidable challenges and immense pressure in managing the COVID-19 pandemic, primarily due to the constraints posed by limited medical resources. The Chinese government implemented policies such as “Pandemic Lockdown” or “Dynamic Cleaning” at both the central and local levels, and the pandemic persisted for three years in China.¹ In November 2022, the number of the nucleic acid testing positives increase rapidly again in China.² However, in December 2022, the Chinese government issued new policies regarding COVID-19, including the cancellation of nucleic acid testing as the only green pass for the public during the three-year period. This has led the public to perceive that the Chinese government has ended all prevention and control policies against COVID-19. Despite China's unique population characteristics, medical conditions, economic situation, and governance structure, which differ from developed countries, its policies in response to COVID-19 can provide valuable references for other developing countries around the world and may offer insights and lessons for future pandemic prevention and control efforts.

Therefore, in this study, our focus is to analyze the policies issued by the Chinese government during the COVID-19 pandemic. Instead of qualitative analysis of policy influence (whether the policies are better or worse) (Zhu et al., 2022), text mining and network analysis techniques are employed to quantitatively visualize the evolution of China's policies against COVID-19. The process of policy evolution involves the government developing regulations by considering the unique features and requirements of social, economic, scientific, and technological activities at various points in time. This aspect plays a crucial role in public policy research, as highlighted by Yang et al. in 2020. We specifically focus our examination on the key elements of policies, namely the policy targets and policy tools. Policy targets represent the policymakers' intentions in textual form, while policy tools are the political actions and behavioral approaches employed to achieve the policy targets (Huang et al., 2018). Policy tools serve as instruments to facilitate the attainment of policy targets and form the core contents of policies. Hence, we aim to mine and analyze the evolution of China's policies from the perspectives of policy targets and policy tools.

This study is organized as follows: Section 2 provides a review of related works, while Section 3 presents our research design and methods, including data collection and processing, extraction of policy targets, and identification of policy tools. Section 4 presents the evolution of policy targets, policy tools, and the co-evolution of policy targets and policy tools. In Section 5, we analyze the outcomes of our research and derive conclusions based on the findings. Section 6 provides implications of our findings.

Related works

Policy document mining

Policy documents serve as carriers of policy and play a crucial role in governance and blueprint planning. Scholars use policy documents as a mean to explore policy contents and policy processes (Jung and Park, 2015; Wu, 2019). Quantitative research or mining of policy documents has emerged as a new direction in public policy studies (Simonofski et al., 2021). However, policy document mining is distinct from traditional scientific text mining, such as papers and patents, as accurately mining the content of policy text using bibliometric methods can be challenging (Huang et al., 2021). Previous research has mainly focused on policy topic mining, policy issue ministry network analysis, policy diffusion, policy tool/instrument analysis, among others. For example, Huang et al. (2018) proposed a policy target-policy instrument pattern to analysis the policy change. Wang and Zhang (2022) identified high-frequency words in policy documents as policy topics, constructed a policy topic network to explore the antecedents of policy evolution. Li et al. (2022) developed a three-dimensional analysis framework of policy text content to uncover issues in new online ride-hailing policies. Zhang et al. (2022) used author-topic model and LDA to mine the correlation between policy-issuing agencies and policy topics. Wang et al. (2023) propose a discourse parsing framework to specify the functional structure and semantic system of Chinese government document.

Policy targets and policy tools are two crucial dimensions of policy document mining. However, little prior research has paid attention to their co-evolution, especially concerning COVID-19 policies. For instance, in a study conducted by Wang (2021), the influence of national culture and government policies on the implementation of social distancing measures to combat COVID-19 was examined. The findings revealed that the strictness of government policies had a more significant impact on social distancing practices compared to national culture. It underscores the pivotal and proactive role played by government policies in the prevention and control of pandemics. Wu et al. (2021) analyze the policy topics, policy-issuing agencies, and keywords based on the 366 policies against COVID-19, revealing the patterns of EPC in China, but these findings only show a partial microcosm of the EPC by April 2020. While Zhu et al. (2022) begin to analyze of the policy tools employed during different phases of China's COVID-19 outbreak. However, their study did not provide comprehensive quantitative statistics on these policy tools and did not pay attention to the co-evolution of policy targets and policy tools.

Thus, our research aims to bridge this gap by utilizing text mining techniques to extract policy targets and identify specific policy tools from COVID-19-related documents in China. Additionally, we employ quantitative analysis methods to explore the evolution of China's policies in response to the pandemic.

Policy target

Policy targets are key elements of policy documents that articulate the purpose and intention of policy-making. They are crucial for improving the implementation of policy actions. Relying solely on long-term targets makes it challenging in identifying and nurturing early contributions, leading to insufficient emphasis on the crucial initial actions required to drive long-term transformation (Lester and Neuhoff, 2009). Therefore, it is essential to have a mix of long-term, short-term, and specific policy targets to address policy issues effectively. In contrast to policy goals, which are often abstract and qualitative, policy targets are more specific and quantitative, offering a clear direction for policy action (Tuominen and Himanen, 2007). In this context, the policy target cannot be succinctly captured in one or two simple words or phrases. And, policy targets are not static or immutable, as they are shaped by the dynamic characteristics of the policy context and evolve with time and situation. Hence, in-time and automated extraction of policy targets from policy documents is crucial for policy analysis and supervision.

The extraction and analysis of policy targets can provide valuable insights into the intricate dynamics of policy design and implementation. It can shed light on the strategic intentions of policy-issuing agencies and uncover shifts in national strategic planning (Yang and Huang, 2022). Such analysis enables a deeper understanding of the underlying factors that shape policies and their implications for effective governance. For example, Huang et al. (2018) introduced a bibliometric approach aimed at identifying the core policy targets based on a policy target-policy instrument pattern. Their approach involves using a network to connect policy targets and policy instruments and computing their eigenvector centrality to identify the core node. As an illustration, Yang et al. (2020) utilized China's information technology policies to construct networks based on co-occurring policy target keywords, allowing for an analysis of the progression of policy targets over discrete time intervals. By utilizing policy documents, the extraction and analysis of policy targets can serve to enhance the public's comprehension of policies in a more straightforward and easily accessible manner.

For COVID-19 policies, policy targets pertain to the direction of prevention and control. In the context of COVID-19 policies, if all efforts are directed toward achieving “Zero Clearing” without considering economic losses and pressures, then the policy targets must exclusively focus on prevention and control. The policy targets of Chinese policies against COVID-19 can directly and objectively reveal the purpose and intention of the Chinese government. Therefore, the objective of this study is to extract policy targets from policy documents, aiming to illuminate the thinking and considerations of the Chinese government in formulating their policies.

Policy tool

Policy tools, also known as policy instruments, are instruments designed to encourage or enable people to take actions they might not have otherwise taken, or to overcome obstacles to policy-relevant actions (Schneider and Ingram, 1990). Policy tools play a crucial role in public administration. Based on the policy tool theory, governmental policy actions can be regarded as explicit objects, comparable to formal legal tools, rather than an extensive amalgamation of management activities and processes (De Bruijn and Hufen, 1998). Within implementation systems, diverse mechanisms and devices are employed to attain policy targets, and the policy tools methodology offers a conceptual framework for assessing the intricacy of such systems and forecasting outcomes (Blair, 2002). Policy tools have been an important dimension of exploration and promotion of policies, as exemplified by studies such as Xu et al. (2022), which investigated the relationship between environmental policy tools and economic development, examined the mediating effect of public health. Chai et al. (2020) also examined the structure and function of China's environmental policy based on policy tool theory. One of the most fundamental and significant aspects of policy tools research is the classification of policy tools, namely how to identify the category of policy tools used in various policy contexts.

Various scholars have proposed classification frameworks for policy tools over the years. For instance, Rothwell and Zegveld (1984) summarized 12 specific categories of policy tools, including public enterprise, scientific and technical support, education, information, financial, taxation, legal regulation, political, public services, commercial, and overseas agent, which could be further grouped into three broad categories: supply-oriented policy tools, environmental oriented policy tools, and demand-oriented policy tools. Schneider and Ingram (1990) identified five broad categories of policy tools: authority, incentive, capacity-building, symbolic and hortatory, and learning. Howlett et al. (2009) proposed three broad categories: command control tools, economic incentive tools, and social autonomy tools, along with eight specific categories. Bemelmans-Videc et al. (2011) summarized three categories: carrots, sticks, and sermons. Despite more than 40 years having passed, the policy tool classification framework proposed by Rothwell and Zegveld remains the most widely used framework and has been applied in various domains (Huang et al., 2018; Zhu et al., 2022). For example, Lin et al. (2013) used this framework to compare innovation policy tools between China and the USA. Qin et al. (2020) analyzed the evolution of China's marine ranching policy from the perspective of policy tools. Kuo and Shyu (2021) take the innovative policy tools proposed by Rothwell and Zegveld to compare the policies of blockchain technology between the USA and China.

Despite existing research on policy tools, several shortcomings persist. Primarily, previous studies manually identified policy tools, a time-consuming and labor-intensive process. Handling large volumes of policy data becomes especially tedious and cumbersome, hindering the efficient processing of extensive policy tool identification by data coders. This limitation has led to the frequent use of small identification samples in prior studies. To address this gap, our study introduces an active learning process for the automatic identification of policy tools. This process is based on an enhanced classification framework derived from Rothwell and Zegveld's, as outlined in Table 1. The classification framework integrates classical and recent research on policy tools, comprising three main categories: supply-oriented policy tools, environmental-oriented policy tools, and demand-oriented policy tools.

Table 1.

The classification framework of policy tools.

Policy tool		Description
Supply Oriented	Cultivation of Talent	Enhancing talent assurance through measures like talent training, talent acquisition, and talent development planning.	Qin et al., 2020; Kuo and Shyu, 2021
	Funds Support	Providing direct financial support from the government for development initiatives, including investment in research and development (R&D), science and technology funds, and financial subsidies.
	Technical Support	Policies to provide techinical support to promote the development.
	Public Enterprise	Policies mobilizing the state-owned enterprises and institutions, joint with private enterprises.
	Infrastructure Development	Enacting policies that facilitate infrastructure development.
Environmental Oriented	Legal Regulation	Establishing laws and regulations focused on pandemic prevention and control.	Lin et al., 2013; Kuo and Shyu, 2021
	Target Planning	Policies about the plan or strategy.
	Financial Support	Implementing policies to incentivize and attract financial and derivative institutions to offer support such as loans, guarantees, venture capital, and other forms of assistance.
	Tax Preference	Implementing tax measures, such as tax exemptions and reductions for specific projects, R&D tax credits, capital gains tax exemptions, and personal tax allowances.
	Political Support	Developing strategic plans or national programs, regional policies, utilizing think tanks and engaging in public consultations for policy development, and political and legal system improvement.
Demand Oriented	Government Procurement	Facilitating government purchases and contracts at the central or local level, including R&D contracts and transactions via government procurement.	Melnikov and Karelin, 2021;Yao et al., 2021;
	Service Outsource	Government's R&D plans distributed to enterprises or industry university research collaboration organization.
	Demonstration Project	Establishing industry-university research demonstration bases, demonstration projects, or major demonstration projects.
	Guidance and Encouragement	Measures to encourage and lead the public into related action.

Supply-oriented policy tools emphasize the government's utilization of professional talents, technology, funding, and public services to promote specific political activities. This category includes five subcategories: cultivation of talent, fund support, technical support, public enterprise, and infrastructure development (Qin et al., 2020; Kuo and Shyu, 2021). Talent cultivation involves the government's training of relevant professionals through education and training, ensuring a skilled workforce. Funding support entails the government providing special funds to support pertinent work. Technical support involves the government providing technological assistance to advance relevant projects. Public enterprise refers to the government supplying basic support services to ensure the smooth progress of work. Infrastructure development entails the government offering material support through the construction and improvement of infrastructure.

Environmental-oriented policy tools encompass five subcategories: legal regulatory, target planning, financial support, tax preference, and policy support (Lin et al., 2013; Kuo and Shyu, 2021). Legal regulatory actions involve the government strengthening norms and supervising relevant work through mandatory measures such as laws and regulations. Target planning includes the government's overall planning or description of relevant work through administrative means to achieve goals and formulate specific implementation plans. Financial support refers to the government providing necessary financial support to create a favorable policy environment for relevant work. Tax preference involves the government formulating tax incentives such as cuts, subsidies, and deferred payments to stimulate market vitality. Policy support encompasses the government providing necessary policy support to create a favorable policy environment for relevant work.

Demand-oriented policy tools consist of four subcategories: government procurement, service outsource, demonstration projects, and guidance and encouragement (Melnikov and Karelin, 2021; Yao et al., 2021). Government procurement entails central or local governments purchasing materials or services from third-party enterprises or profit organizations using financial funds. Service outsource involves the government collaborating with social capital to promote the development of relevant work or directly outsourcing specific tasks to relevant enterprises and companies. Demonstration projects refer to the government promoting the implementation and development of relevant work through the construction of demonstration projects. Guidance and encouragement involve the government using measures to encourage and guide (reward, commendation) the public's relevant behavior to promote the development of relevant work.

The description of each type of policy tool are provided in Table 1.

Research design and methodology

The key steps of our research design and methodology are outlined in Figure 1, as follows:

Figure 1.

The policy target and tool co-evolution mining framework about COVID-19.

Data collection and descriptive analysis

The policy documents related to COVID-19 were gathered from the PKULaw database,³ a highly authoritative and comprehensive legal database in China. Within this database, we utilized a specialized sub-database dedicated to “Epidemic Prevention and Control” (EPC) as an example. To ensure the representativeness and validity of the policy sample, only national policies issued by the central government were considered, and local policies were excluded. The study focused on policies published between December 27, 2019, and September 10, 2022, to ensure that the data specifically pertained to COVID-19 and its prevention and control measures.

The collected data underwent a manual screening process based on the following criteria:

The policy document had to be in the form of a law, regulation, circular, or another official document representing government policy.

The main content of the policy had to be closely related to COVID-19.

Following the screening process, a total of 1154 policy documents were obtained and uploaded to github.⁴ Table 2 displays the top 30 government ministries that issued policies in response to COVID-19, along with the corresponding number of policy documents they issued.

Table 2.

The top 30 ministries/department with highest number of issued policies.

Ministry/department	Number of policy documents	Ministry/department	Number of policy documents	Ministry/department	Number of policy documents
National Health Commission	180	Ministry of Industry and Information Technology	39	Supreme People's Procuratorate	24
Ministry of Finance	129	State Administration for Market Regulation	38	National Administration of Traditional Chinese Medicine	22
Ministry of Transport	97	General Administration of Customs	37	State Administration of Taxation	21
the joint prevention and control mechanism on coping with the novel coronavirus	73	Civil Aviation Administration of China	37	National Healthcare Security Administration	20
Ministry of Commerce	66	Ministry of Culture and Tourism	32	National Medical Products Administration(changed)	19
Ministry of Human Resources and Social Security	64	Ministry of Public Security	30	People's Bank of China	19
Ministry of Education	60	Ministry of Agriculture and Rural Affairs	30	China Banking and Insurance Regulatory Commission	19
Ministry of Civil Affairs	53	Ministry of Housing and Urban-Rural Development	28	National Intellectual Property Administration	16
National Development and Reform Commission	47	Ministry of Justice	24	Ministry of Ecology and Environment	14
integrated group of Joint Prevention and Control Mechanism on coping with the novel coronavirus of the State Council	40	Supreme People's Court	24	National Immigration Administration	13

According to the publication “Fighting COVID-19 China in Action” by the State Council Information Office of the People's Republic of China,⁵ the COVID-19 epidemic is categorized into four distinct temporal stages known as “stage 1,” “stage 2,” “stage 3,” and “stage 4.” This categorization was established by the central government. Table 3 provides a detailed overview of these stages and the distribution of China's policy against COVID-19.

In Stage 1, as the number of confirmed cases rapidly escalated and the understanding of the virus deepened, the National Health Commission of China enacted its first policy regarding the prevention and control of the disease on January 20, 2020. This marked the formal initiation of the country's efforts to manage the spread of the epidemic. Over this stage, the number of relevant policy documents significantly increased, with a total of 344 policies being issued. These documents comprised 29.8% of the total policy texts produced during the COVID-19 pandemic.

In Stage 2, significant progress was made in the prevention and control efforts. One of the most notable milestones during this stage was the decrease in the number of new domestically transmitted cases to single-digit figures. In response, the Central Committee of the Communist Party of China made a decisive decision to reconcile the targets of controlling the spread of the virus with the continued development of the economy and society. As a result, a gradual and organized restart of work and production activities was initiated.

In stage 3, significant progress was made in limiting the spread of the virus within the country. The Central Committee of the Communist Party of China established a comprehensive strategy for preventing the importation of new cases from abroad, as well as for mitigating the risk of a resurgence of the disease domestically. On March 18, 2020, China recorded its first day with zero new locally acquired cases of COVID-19.

In stage 4, the domestic transmission of the virus was largely limited to isolated cases. Effective measures were implemented to control the importation of cases from abroad, resulting in a transition to a normalized state of EPC. Correspondingly, the implementation of relevant policies and regulations gradually increased.

Table 3.

The stages and distribution of China's policy against COVID-19.

Stage	Time frame	Stage feature	Policy number
1	2020.1.20–2020.2.20	Preliminary containment	344
2	2020.2.21–2020.3.17	Initial control	182
3	2020.3.18–2020.4.28	Overall control	143
4	2020.4.29–2022.9.10	Regular prevention and control	485

Policy target extraction

A “policy target” denotes the desired outcome and purpose underlying a policy-making action, representing the primary issue that the policy seeks to resolve. Identifying policy targets can often be challenging as they are frequently implicit. To tackle this problem, we employ a policy target identification procedure (Yang et al., 2020), which harnesses natural language processing techniques and regular expressions. The procedure is outlined as follows:

Initially, the introductory paragraph of each policy document is located. The end of the introductory paragraph is determined by identifying the appearance of structures such as “。+\n” or “：+\n” in the policy text.

Subsequently, policy targets within the introductory paragraphs are identified by utilizing a verb/noun thesaurus generated from common policy expressions. In Chinese policy documents, policy targets are often presented in the form of “in order to + verb + noun phrase.” For instance, in the sentence “in order to promote the transformation of scientific and technological achievements,” the policy target is “promoting the transformation of scientific and technological achievements.” By recognizing this specific structure and verifying that the words within the structure align with the pre-constructed verb/noun thesaurus, which is developed using expert knowledge and extensive examination of policy texts, it can be determined that a policy target has been successfully identified. The analysis involves summarizing the common nouns and verbs that appear in policy targets, along with their combinations. Table 4 provides a partial overview of the “verb/noun thesaurus.” In cases where the policy target could not be accurately extracted from very short policy texts, manual reading was employed to supplement the extraction process.

Clean the initially extracted policy targets by eliminating stop words and consolidating synonymous terms, among other adjustments, and so on.

Identify the core terms within policy target words. To do this, we used TF-IDF to compute the importance of each term in the cleaned policy targets words/phrases. The more important the term, the more core it is considered. The results were then manually reviewed and modified, and core terms were abstracted and regarded as the final policy target keywords. The TF-IDF weight was calculated as follows:

T F - I D F (t, d) = T F (t, d) \times I D F (t)

(1)

where TF(t, d) represents the frequency of term t in document d, and IDF(t) represents the inverse document frequency, which was used to determine the importance of term t in expressing a semantic dimension. It is calculated as follows:

I D F (t) = \frac{\log total number of documents}{number of documents containing t + 1}

(2)

After extracting the policy target keywords, we constructed policy target keyword co-occurrence networks and identified the core policy target keywords in each time period based on eigenvector centrality. Eigenvector centrality is a technique used to compute the significance of a node in a network, based on the PageRank. Eigenvector centrality was chosen because it comprehensively considers both a node's direct connections and the significance of its neighbors, offering a holistic assessment of a node's influence within the network. Compared to other centrality measures, it better reflects the overall structure and complex relationships in the network. It is defined as follows:

E_{v} = \frac{1}{λ} \sum_{k \in P (v)} E_{k} = \frac{1}{λ} \sum_{k \in G P} A d_{v, k} E_{k}

(3)

where P(v) is the set of neighbors of node v, λ is a constant representing the eigenvalue, Ad_v,k is the adjacency value between node v and k.

Table 4.

Verb/noun thesaurus (partial).

Verb	Noun
implement, provide, reduce, carry out, safeguard, maintain, promote, increase, support, strengthen, protect, curb, win, mitigate, improve, accelerate, enhance, promote, cope, encourage, guide, strengthen, play, help, establish, help, raise, reduce, regulate, prevent, control, simplify	spirit, deployment, service, risk, thought, work, principle, safety, stability, development, requirement, strength, recovery, management, health, instruction, supply, momentum, war, prevention, control, burden, integrity, progress, construction, capacity, guarantee, resumption, impact, resumption, epidemic, initiative

Policy tool identification

In the policymaking process, policy documents often encompass a variety of policy tools, presenting a challenge in accurately identifying and parsing them. To tackle this issue, we employed a trained policy tool classification model based on the framework outlined in Table 1. This model incorporates pretrained WordBERT-ZH and BiLSTM. WordBERT-ZH, an enhanced version of BERT tailored for Chinese text classification, has demonstrated superior accuracy compared to other models, including BERT-Base, RoBERTa-Base, WoBERT, and MarkBERT (Feng et al., 2022). The BiLSTM functions as a classifier, establishing the relationship between the embedding of policy text and the policy tool categories.

This policy tool classification model underwent training on 8910 manually labeled samples by the authors, making it the most accurate automatic classification model for Chinese policies proposed thus far (Huo et al., 2023). While its accuracy in COVID-19 policies is uncertain, steps were taken to ensure and enhance accuracy in this specific dataset. Following active learning theory, we incorporated manually labeled samples derived from COVID-19 policies. Active learning is a mechanism that minimizes the need for human annotation in training datasets, selecting the most informative cases to achieve superior performance compared to passive supervised learning algorithms (Figueroa et al., 2012). Active learning has been successfully applied across various domains, including text classification and image classification (De Angeli et al., 2021).

The identification of policy tools is carried out through the following steps:

First, we treat each policy paragraph as one policy unit, as considering one paragraph as one policy unit ensures that the statistics about policy tools are comparable and maintains semantic integrity, following the classical process of policy unit standardization (Yang and Huang, 2022). We then divide all policy documents into policy units, resulting in 8910 policy units related to COVID-19 policies.

Second, based on the model trained in digital economic policy and data governance policy, we classify the policy tools for each policy unit from the COVID-19 policy. To test the accuracy of the classification, we randomly selected 100 policy units and assigned a domain expert proficient in policy tool classification to independently encode them manually. By comparing the results of the model encoding and the manual encoding, we calculated Holsti's consistency percentage to assess the reliability of the model. The formula to calculate Holsti's consistency percentage is as follows:

R e l i a b i l i t y = \frac{2 M}{N_{1} + N_{2}}

(4)

where M represents the number of codes that are completely the same in two encodings, and N₁ and N₂ represent the number of codes in the two encodings, respectively.

Finally, we optimized the classification according to the theory of active learning. As shown in Figure 2, we first checked the results of the policy tool classification model. If the accuracy of a certain category is not satisfactory, indicating that it is difficult to classify, we added more labeled samples to create a new training dataset for these categories with low performance. We then trained the model with this new dataset and tested the results again. After 9 rounds of coding and supplementing the training dataset, the final reliability of the first coding result reached 90%, meeting the reliability requirements (80%). In total, we identified 8517 policy tool items from the COVID-19 policies.

Evolution analysis of China's policy against COVID-19

The evolution of policy targets

Utilizing the methodology detailed in Section 3.2, we conducted an analysis of policy documents related to the COVID-19 pandemic. This process led to the identification and extraction of 208 policy target keywords. Subsequently, policy target keyword co-occurrence networks were constructed for each stage based on their relationships in the documents. In these networks, nodes represented policy targets, and edges between nodes denoted co-occurrence in policy documents. Node size was determined by the frequency of occurrence of the policy target, while edge thickness was determined by the frequency of co-occurrence of two policy targets.

Figure 2.

The process of active learning.

To glean insights into the historical evolution of COVID-19 policies, we calculated eigenvector centrality for each node in the network. The policy target keywords with the highest eigenvector centrality are listed in Table 5. A comparative analysis of the core policy targets in different stages of China's response to the COVID-19 pandemic revealed several key findings:

In Stage 1, as shown in Figure 3, the co-occurrence network showed that the central policy target was “COVID-19 prevention and control,” which had the highest frequency and eigenvector centrality. This core target was closely linked with “safety and health” and “implementation of the central strategy,” which ranked among the top three in terms of eigenvector centrality and co-occurrence frequency.

Figure 3.

China's COVID-19 policy targets co-occurrence network, stage 1.

Table 5.

The policy targets of China's COVID-19 policies (top 10).

No.	Stage 1		Stage 2		Stage 3		Stage 4
No.	Target keyword	Eigenvector centrality	Target keyword	Eigenvector centrality	Target keyword	Eigenvector centrality	Target keyword	Eigenvector centrality
1	COVID-19 prevention and control	1.00	COVID-19 prevention and control	1.00	COVID-19 prevention and control	1.00	COVID-19 prevention and control	1.00
2	Implement of the central strategy	0.76	Economic and social development	0.80	Economic and social development	0.89	Economic and social development	0.79
3	Health and safety	0.60	Work resumption	0.61	Work resumption	0.74	Implement of the central strategy	0.73
4	Implement the spirit of meeting	0.56	Implement of the central strategy	0.56	Implement of the central strategy	0.51	Implement the spirit of meeting	0.59
5	Work resumption	0.56	Implement the spirit of meeting	0.51	Protect legitimate rights and interests	0.50	Health and safety	0.55
6	Strengthen financial support	0.50	Precise and differentiated epidemic control strategies	0.38	Promote industry development	0.44	Prevent the epidemic from rebounding	0.54
7	Personnel and materials supply	0.46	Personnel and materials supply	0.37	Ensure stability in foreign trade and foreign investment	0.38	Defending externally against importation	0.54
8	Prevent the spread of epidemic	0.44	Health and safety	0.34	Maintain economic and social order	0.37	Maintain economic and social order	0.50
9	Medical treatment	0.38	Stabilize employment	0.32	Stabilize employment	0.36	Consolidate prevention and control achievements	0.46
10	Ensure orderly work	0.38	Medical treatment	0.31	Reduce costs and burdens	0.34	Prevent the spread of epidemic	0.44

During this stage, the central government emphasized the critical importance of controlling the spread of the epidemic and safeguarding public health, surpassing market economy concerns. The policies aimed to ensure a steady supply of essential goods and medical supplies while preventing the virus's spread through unified leadership and coordinated efforts. Simultaneously, the government took proactive measures to facilitate the resumption of work and production, providing financial assistance and ensuring the smooth operation of various industries. However, market activation and support initiatives primarily focused on the production of medical supplies and essential commodities, with the primary goal of aiding virus containment efforts.

In conclusion, it is evident that during this stage, the government prioritized efforts to combat the COVID-19 epidemic, placing it ahead of other market-related objectives.

In stage 2, as illustrated in Figure 4, the core policy target remained “COVID-19 prevention and control.” However, there was a shift in the combination of core policy targets, which now included “Economic and social development.” This indicates a shift in the government's priorities, moving beyond the initial emergency response phase toward addressing economic and social concerns. The emphasis now lies on coordinating epidemic control measures with broader macro-level goals of economic development. Despite this shift, the government's priority still remained on EPC, with core targets related to this objective including “Precise and differentiated epidemic control strategies,” “Personnel and materials supply,” “Health and safety,” and “Medical treatment.”

Figure 4.

China's COVID-19 policy targets co-occurrence network, stage 2.

It is worth noting that a new target, “Precise and differentiated epidemic control strategies,” has emerged in this stage, ranking sixth in Table 5. This indicates that the government's policies are now more focused on implementing precise and scientific COVID-19 prevention and control measures, transitioning from blanket lockdowns to graded, zoned, and targeted approaches. Furthermore, the target of “Public opinion control and public guidance” has become more prominent, even though it does not appear in the top ten list. This highlights the Chinese government's emphasis on shaping public opinion and guiding public behavior to support the implementation of measures.

During this period, the target of “Stabilize employment” has made its debut appearance in the top ten list, and it appears to be nearly as important as the second and third targets of “Economic and social development” and “Work resumption.” This underscores that the government's priorities during this period are increasingly focused on stabilizing employment and restoring order, in order to ensure a solid foundation for future economic development.

In stage 3, as depicted in Figure 5, there is a noticeable trend toward a more coordinated and unified approach in formulating macro-level policy targets. Table 5 shows that the difference in eigenvector centrality between the two core targets, “COVID-19 prevention and control” and “Economic and social development,” has reduced. This shift toward economic and social development is further emphasized by specific policy targets that focus on promoting economic development through various means, such as laws, foreign trade, and market environment. These targets include “Work resumption,” “Protecting legal rights and interests,” “Promoting industrial development,” “Ensuring stability in foreign trade and foreign investment,” “Stabilizing employment,” “Maintaining economic and social order,” and “Promoting consumption.” This highlights the paramount importance placed by the government on social and economic development. Additionally, “Public opinion control and public guidance” appears more frequently, indicating that public opinion has been a crucial social issue that has impacted the implementation of COVID-19 prevention and control policies. Public opinion control and public guidance are always important when facing challenges from uncertainty, although public opinion in China has been well managed and rumors have been promptly addressed by the government.

In stage 4, as illustrated in Figure 6, there is a heightened policy relevance, with a more discernible clumping and interconnectedness among policy targets, forming a systematic planning system that concurrently facilitates economic development and EPC. The core targets in this stage are almost consistent with those in the previous stage, such as “COVID-19 prevention and control” and “Economic and social development,” which highlights the Chinese government's commitment to integrating EPC with economic and social development. The policies during this stage emphasize the consolidation of prevention and control outcomes, such as “Preventing the resurgence of the epidemic,” “Preventing the importation of the epidemic,” “Consolidating the outcomes of prevention and control,” and “Preventing the spread of the epidemic.”

Figure 5.

China's COVID-19 policy targets co-occurrence network, stage 3.

Figure 6.

China's COVID-19 policy targets co-occurrence network, stage 4.

Additionally, three targets deserve special attention during this stage, namely “Traffic control,” “Nucleic acid testing,” and “Financial investment.” Although they are not ranked in the top 10, they have a high occurrence frequency, especially “Financial investment,” which exhibits a high frequency of co-occurrence with the other two policy targets. Traffic control represents a specific measure aimed at mitigating the transmission of the epidemic, nucleic acid testing facilitates early detection, treatment, and control of the epidemic, while financial investment reflects the government's support toward related endeavors. In other words, during the last stage of COVID-19, the Chinese government took significant measures, including traffic control, to prevent the spread of COVID-19, even though other countries had not implemented such measures three years ago. This serves as a valuable lesson for China's future pandemic prevention and control efforts.

In summary, China's EPC policy targets have evolved from “Emergency prevention and control,” “Prevention and control while stabilizing the economy,” and “Economic and social development,” to “Consolidating prevention and control results and promoting economic development.” The Chinese government has consistently emphasized central coordination and the sub-targets of EPC, which have evolved from treatment, material supply, and prevention of epidemic spread, to early detection and prevention, and to prevention of epidemic importation and resurgence. These targets are specific, up-to-date, and constantly optimized. However, China seems fall into a self-paradox between pandemic prevention and control and economic and social development, as achieving both targets simultaneously may be challenging without effective vaccines and new measures to address COVID-19.

The evolution of policy tools

The policy tool classification model was utilized to categorize the policy units, resulting in the identification of 8517 basic policy tools, spanning three categories and 14 s-categories. As depicted in Table 6, there is a noticeable disparity in the distribution of policy tools, with 68.8% classified as environmentally oriented, 23.8% as supply-oriented, and 7.3% as demand-oriented. This suggests that the government primarily employs “Top-down” measures to manage the spread of the epidemic, by creating a favorable policy environment and directly supplying resources and services, while disregarding the pull effect from the demand side.

Table 6.

Distribution of policy tools.

Policy tool		Stage 1	Stage 2	Stage 3	Stage 4	Total	Proportion (%)
Supply Oriented	Cultivation of Talent	2.8	3.7	5.9	2.3	266	3.1	23.8
	Funds Support	1.3	1.8	1.3	0.7	91	1.1
	Technical Support	1.9	1.3	2.1	1.0	119	1.4
	Public Enterprise	14.1	19.4	11.3	9.2	1018	12.0
	Infrastructure Development	5.2	3.0	4.7	8.2	538	6.2
Environmental Oriented	Legal Regulation	25.4	19.5	25.7	28.3	2229	26.2	68.8
	Target Planning	30.2	30.9	28.7	37.6	2871	33.7
	Financial Support	3.3	3.0	3.9	3.5	292	3.4
	Tax Preference	3.1	5.6	1.4	1.9	220	2.6
	Political Support	4.4	4.8	2.6	1.9	251	2.9
Demand Oriented	Government Procurement	0.1	0	0	0.2	10	0.1	7.3
	Service Outsource	1.8	2.2	2.6	1.0	133	1.6
	Demonstration Project	0.7	1.4	5.4	0.8	129	1.5
	Guidance and Encouragement	5.6	3.6	4.3	3.5	350	4.1

Firstly, the utilization of supply-oriented tools is relatively limited. Among these policy tools, public enterprise constitutes the largest proportion (12.0%), followed by infrastructure development (6.2%), and talent cultivation (3.1%). This indicates that the government prioritizes strategies that directly support EPC measures, such as infrastructure construction and service optimization, and provides impetus for EPC through human, material, and service support. However, fund support and technical support tools only account for 1.1% and 1.4% respectively, suggesting that more attention needs to be given to fund allocation and application of science and technology in policy formulation. In other words, the Chinese government has not adequately addressed these areas from the perspective of public policy, despite the challenges posed by COVID-19. The shortage of medical resources cannot always be attributed as the sole reason for inadequate pandemic prevention and control. Strengthening these areas through increased fund support and technical assistance is imperative.

Secondly, the government heavily relies on environment-oriented policy tools throughout the four stages. The most frequently used policy tools are legal regulation and target planning, accounting for 26.2% and 33.7% respectively. These two types of policy tools consistently maintain high proportions across all stages, signifying the government's continuous efforts in building and optimizing the policy implementation environment, to ensure the smooth execution of EPC measures, as well as monitoring the progress of EPC and resumption of production. However, financial support, tax incentives, and political support are relatively minor in proportion, yet they contribute to maintaining stability in the overall economic environment, reducing pressure on enterprises to resume operations, and aiding economic recovery. Therefore, the government should also consider issuing more varied policy tools related to financial support, tax incentives, and political support in detail, rather than solely relying on general legal regulation.

Lastly, the use of demand-oriented policy tools is insufficient across the four stages. Demand-based policy tools account for the lowest ratio among the three types of policy tools and are severely inadequate and lacking in utilization. This indicates that the Chinese government currently focuses primarily on boosting the epidemic response through increasing supply and creating a favorable policy environment, but overlooks the need to stimulate development from the demand side. Demonstration projects, service outsourcing, and government procurement policy tools all account for less than 2%, with government procurement policy tools only accounting for 0.1%. This suggests that the government needs to introduce more measures to attract social and market forces to participate in the epidemic response and economic development. Although the Chinese government has implemented service outsourcing and demonstration projects, such as collaborating with nucleic acid testing companies and constructing Huoshenshan Hospital and shelter hospitals, these efforts are not adequately reflected in policies. Apart from government-initiated special projects, the government should also issue more demand-oriented policy tools to actively engage and lead contributions from society and the market.

In conclusion, the COVID-19 policy tools employed in China exhibit a tendency toward emphasizing environmental creation over demand stimulation, relying on government intervention over market forces, prioritizing regulation over cooperation, and favoring coercion over guidance.

The co-evolution of policy targets and policy tools

We conducted a cross-comparison analysis of policy tools and policy targets to evaluate the effectiveness of combining different analytical dimensions in achieving policy targets. The proportion of policy tools used for the top ten core policy targets in each stage is calculated, and the results are presented in Figures 7–10.

Figure 7.

Distributions of policy tool and policy target cross-cutting dimensions, stage 1.

Figure 8.

Distributions of policy tool and policy target cross-cutting dimensions, stage 2.

Figure 9.

Distributions of policy tool and policy target cross-cutting dimensions, stage 3.

Figure 10.

Distributions of policy tool and policy target cross-cutting dimensions, stage 4.

The results indicate that the ten policy targets in each stage generally cover all the second-category policy tools, but some policy tools are underutilized and weak in achieving the refined targets. In stage 1, environmental-oriented tools and supply-oriented tools have high coverage and dominant positions, particularly in target planning, legal regulation, and public enterprise. However, technical support and financial support policy tools have insufficient coverage. Demand-oriented tools have a low coverage rate for most targets, except for guidance and encouragement. Demonstration project tools and government procurement tools do not cover most targets, and the number of supply-oriented and demand-oriented tools used falls short of meeting the personnel and material supply demand.

In stage 2, the public enterprise tool has a higher coverage rate, while the legal regulation tool has a lower coverage rate compared to the first stage. There is an extreme disproportion in the policy tools used in precise and differentiated epidemic control strategies and personnel and materials supply, as these two policy targets use six and nine kinds of tools, respectively. The government procurement tools do not cover all targets. Generally speaking, these policy targets should be equipped with polity tools proportionally. This study notes that the coverage rate of health and safety targets in technical support tools increases from 0% in the first stage to 21.43% in the second stage, indicating a shift in government focus toward using technology such as information technology and medical technology to protect people's health and safety. It's an important and positive optimization in policy making.

In stage 3, the legal regulation tool has a higher proportion than in the second stage. This study finds that the disproportion between policy tools and policy targets is serious, especially with only the coverage rate of “reduce costs and burdens” and “protect legitimate rights and interests” targets in legal regulation tools being higher than 50%. The analysis of the tools' effects on the targets shows that no government procurement tool is used, and the proportion of demand-oriented tools is higher.

In stage 4, environmental-oriented policy tools have a dominant proportion, followed by supply-oriented tools, while demand-oriented tools are particularly scarce. The second-category tools under the environmental-oriented tool are unbalanced, with insufficient use of financial support, political support, and tax preference. For the “prevent the spread of epidemic” target, the proportion of public enterprise is much higher than in the first stage.

In conclusion, this study reveals that the proportion and operability of policy tools used in China's policy against COVID-19 to address policy targets are inadequate, resulting in an unreasonable matching choice that directly affects the policy's implementation effectiveness and achievement of its targets.

Discussion and contribution

This paper is arguably the first systematic mining and review about the COVID-19 policies issued by the Chinese government. Our policy target and tool co-evolution mining framework provides a new framework for policy document mining. Based on this framework, not only the evolution of policy targets and policy tools are exposed respectively, but also the co-evolution of policy target-policy tool is revealed visually. In this framework, the policy tools are identified via large language model automatically according to the authoritative policy tool categories, rather than Huang et al. (2018) use an instrument thesaurus that summarized by themselves and regard each verb or noun as a policy instrument node. Thus, the policy target-tool co-evolution mining framework we proposed is a novel framework compared to the pattern proposed by Huang et al. (2018) and Wang and Zhang (2022).

Based on this new framework, this study understood the evolving concerns of government departments and the situation-specific strategies they have employed in their efforts to prevent and control the spread of the virus. The results will provide valuable lessons for future EPC efforts, not only in China but also in other developing countries. Ultimately, this research will contribute to the development of effective and evidence-based public health policies that can better respond to the challenges posed by infectious disease outbreaks.

In terms of policy targets, the Chinese government's focus during the pandemic has shifted across different stages, reflecting the changing characteristics of the government's response. The initial stage emphasized COVID-19 prevention and control, with emergency control measures implemented to prevent the spread of the virus and ensure basic livelihoods. Subsequent stages saw a combination of epidemic prevention and economic development, with flexible and tiered measures adopted to control the spread of the virus. In the later stages, efforts were made to balance epidemic prevention with economic and social development, with an increased emphasis on nucleic acid testing.

Throughout the response process, the government prioritized implementing central deployments and spiritual indicators, reflecting the advantage of unified leadership in China. However, some shortcomings were also revealed, such as shortages, uneven distribution, and inefficient use of emergency health resources, as personnel and materials supply has been one core policy tool and important task of the epidemic response in the first two stages. China's large population and relatively scarce medical resources exacerbate the problem of uneven distribution of medical resources, with quality medical resources mainly concentrated in first- and second-tier cities. In addition, people's lack of understanding of the actual situation of the epidemic caused panic, leading to the hoarding of large amounts of health resources, which resulted in waste of health resources and forced the government to build more health resources than needed, causing waste of financial and human resources (Li et al., 2020a, 2020b).

Notably, the policy target of nucleic acid testing emerged only in the later stage of the pandemic and was not included in the top ten central targets. Medical-related targets, such as personnel and materials supply and medical treatment, were the dominant policy targets throughout the pandemic, indicating that China's public health resources were primarily utilized for medical treatment. However, in preventive areas such as nucleic acid testing, resources were inadequate for the large population. Information disclosure-related targets were not prioritized, and public health information disclosure was not always timely, indicating a need to enhance the scope and degree of information disclosure.

In terms of policy tools, China's response to the COVID-19 pandemic has primarily relied on environment-oriented policy tools, with relatively limited application of supply-oriented policy tools and severe lack of demand-oriented policy tools. These findings are consistent with the research conducted by Qin et al. (2020). Specifically, legal regulation and target planning tools have accounted for a significant proportion of the policy tools used, exceeding 50%, while the utilization of financial support, technical support, government procurement, service outsource, and demonstration project has been relatively low, accounting for less than 2%. This indicates that the government needs to optimize the application of environment-oriented and supply-oriented policy tools, especially emphasis on leveraging market forces, social organizations, and public engagement to enhance diseases resistance. Of particular concern is the relatively weak level of financial support, with the government's related policies involving funds support in supply-oriented policy tools, financial support and tax preference in environment-oriented policy tools all exhibiting low levels of involvement. Among them, funds support accounted for only 1.1%, while financial support and tax preference accounted for only 3.4% and 2.6%, respectively. China's government health expenditures as a percentage of GDP have historically been very low, with less than 1% from 2000 to 2007 and reaching 2.16% for the first time in 2020. This proportion is lower than the average of developed countries such as Germany, the United Kingdom, and Japan, and also fall short of the level of public health government investment in developing countries (Zhu et al., 2022). Overall, at the national level, the process of formulating public health policies before and after the outbreak of the pandemic has been characterized by a lack of financial investment and specific financial policies, which has made it challenging to promote the development of the medical system and garner joint response from the public and enterprises in addressing the to the pandemic.

To improve the alignment between policy targets and policy tools, it is crucial to enhance the proportion and functionality of the selected policy tools. Misalignments between policy targets and tools can directly impede the effectiveness of policy implementation and hinder the achievement of policy targets. For instance, during the first and second stages of the pandemic response, only a limited number of policy tools (around 8 or 9) were employed for the policy target of personnel and materials supply. Notably, there was a significant deficiency in policy tools related to encouraging guidance, funding support, and government procurement. Similarly, for the policy target of medical treatment, there were gaps in government procurement tools and insufficient utilization of technical support tools. These inadequacies underscore the need to reevaluate the proportion between policy targets and tools to ensure optimal policy implementation and successful attainment of policy targets.

Implications

Our study highlights some implications for EPC in future. Firstly, for hospitals, there is a need to develop remote medical technology and improve the tiered diagnosis and treatment system to achieve rapid and equitable distribution of medical resources. Modern electronic communication technology can be utilized to integrate existing health resources in China and enhance the health service capacity of primary hospitals. This can help promote equalization of urban and rural health services, overcome spatial and temporal limitations of health resources, and enable rapid and safe allocation of medical resources. Encouraging and guiding private medical institutions to develop remote consultation, diagnosis, education, and learning for community health service centers, with technical support from large comprehensive hospitals, can strengthen the public-private cooperation and technical support needed for an effective tiered diagnosis and treatment system.

Secondly, for disease control and prevention administration, greater attention should be given to timely detection and prevention of epidemics, and a public health awareness of “Prevention is better than treatment” should be established. The Chinese government should increase spending on preventive public health measures, with a focus on prevention and control of major epidemics, vaccine administration, improved virus detection efficiency, and the development of public health emergency systems. This can help prevent outbreaks before they occur and achieve the best prevention and control effects, particularly in the context of major epidemics such as COVID-19.

Thirdly, for official news agencies, information disclosure should be strengthened to facilitate effective epidemic monitoring and response. Sentinel hospitals and communities can be leveraged to strengthen epidemic monitoring in compliance with legal requirements for infectious disease management. Third-party organizations can be entrusted to analyze infection data between different regions and share the results publicly for reference by medical institutions and pharmaceutical companies. Information sharing can be enhanced between comprehensive hospitals and specialized hospitals, as well as between tertiary hospitals and primary hospitals. Collaborative efforts among the government, media, disease control agencies, and industry associations can actively promote science education and health education to improve public awareness of COVID-19 infections.

Fourthly, multi-party participation should be strengthened in EPC efforts. While the government should play a leading role, social participation should be widely mobilized to achieve proportionable distribution of resources, environment, and demand-side policies. Regulatory and target planning should enhance the policy's coercive force while maintaining government's unified leadership, and government's participation and support should continuously enhance demand-pull effects to encourage and guide participation from all sectors of society. Strengthening public-private cooperation and public procurement efforts can facilitate multi-party participation in EPC and resumption of work and production.

Fifthly, greater financial investment in medical care is necessary to support effective EPC measures. Implementing feasible fiscal policies can attract enterprises, social groups, and various social organizations to actively participate in resuming work and production, which is conducive to creating a favorable situation where all sectors of society participate in EPC and resumption of work and production.

Finally, for policymakers, policy tools and targets should be coordinated to improve policy effectiveness. For example, providing financial investment and technical support for medical care, strengthening foreign investment and trade policies, and providing policy support can enhance the effectiveness of the policies in addressing epidemics and promoting public health. Coordinated and well-aligned policy tools can synergistically reinforce each other and contribute to more effective EPC outcomes.

Limitations

Although we propose a new policy target-policy tool co-evolution framework to mine the COVID-19 policy documents and draw lessons from these policy targets and policy tools, the conclusion and evaluation about the COVID-19 of China are all policy-based, which means the policy implementation may also impair the outcome while the policy are well designed. In other words, it's hard to evaluate the all measures that China has taken and the outcome of this whole worldwide pandemic.

The evidence is mined from Chinese policy documents, namely the insights and lessons are only draw from China. This means the experiences may not suit other countries. But we all should remember these polices, and the lesson should be taken into deep consideration. COVID-19 is gone, but epidemics of diseases do not stop.

Footnotes

Acknowledgments

This article was supported by the National Natural Science Foundation of China (Grant Nos. 72004221 and 72374202).

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Chaoguang Huo

Notes

About the authors

Chaoguang Huo is an associate professor at Renmin University of China, the young distinguished scholar at Renmin University of China. He is a visiting scholar at Indiana University. His research interests focus on policy informatics and public health. He has published more than 40 papers in Information Processing & Management, Information Development, Library High Tech, et al.

Xinru Li is a postgraduate student in information science at Renmin University of China. Her research interests focus on public health policy. She is graduated from the School of Public Health.

Chenwei Zhang is an assistant professor at the Faculty of Education in the University of Hong Kong. She obtained her PhD in Information Science from Indiana University, with a PhD minor in computer science. Her research has two streams—one focuses on science of science, which studies the mechanisms underlying the doing of science. The other stream is mainly in data science, where a variety of computational approaches including machine learning and deep learning are applied for knowledge discovery.

Fanfan Huo is an assistant professor at Institute of Scientific and Technical Information of China. Her research interests focus on policy informatics and text mining. She has published more than 10 papers.

References

Bemelmans-Videc

Rist

Vedung

(2011) Carrots, Sticks, and Sermons: Policy Instruments and Their Evaluation. New Brunswick, NJ: Transaction Publishers.

Blair

(2002) Policy tools theory and implementation networks: Understanding state enterprise zone partnerships. Journal of Public Administration Research and Theory 12(2): 161–190.

Chai

Zhang

(2020) Evolution of environmental policy for China’s rare earths: Comparing central and local government policies. Resources Policy 68: 101786.

De Angeli

Gao

Alawad

, et al. (2021) Deep active learning for classifying cancer pathology reports. BMC Bioinformatics 22(1): 1–25.

De Bruijn

Hufen

(1998) The traditional approach to policy instruments. Public Policy Instruments: Evaluating the Tools of Public Administration 242: 11–32.

Dewi

Nurmandi

Rochmawati

, et al. (2020) Global policy responses to the COVID-19 pandemic: Proportionate adaptation and policy experimentation: A study of country policy response variation to the COVID-19 pandemic. Health Promotion Perspectives 10(4): 359.

Feng

Tang

Zhou

, et al. (2022) Pretraining without Wordpieces: Learning Over a Vocabulary of Millions of Words. arXiv preprint arXiv:2202.12142.

Figueroa

Zeng-Treitler

Ngo

, et al. (2012) Active learning for clinical text classification: Is it better than random sampling? Journal of the American Medical Informatics Association 19(5): 809–816.

Gupta

Santosh

Arora

, et al. (2022) Socioeconomic impact due to COVID-19: An empirical assessment. Information Processing & Management 59(2): 102810.

10.

Howlett

Ramesh

Perl

(2009) Studying public policy: Policy cycles and policy subsystems. Oxford: Oxford University Press.

11.

Huang

Yang

(2018) Policy change analysis based on “policy target–policy instrument” patterns: A case study of China’s nuclear energy policy. Scientometrics 117(2): 1081–1114.

12.

Huang

Yang

(2021) Identifying core policy instruments based on structural holes: A case study of China’s nuclear energy policy. Journal of Informetrics 15(2): 101145.

13.

Huo

Wang

, et al. (2023) Policy tools automatic classification based on WordBERT and BiLSTM. Documentation, Information & Knowledge 40(3): 129–138. in Chinese.

14.

Jung

Park

(2015) A semantic (TRIZ) network analysis of South Korea’s “open public data” policy. Government Information Quarterly 32(3): 353–358.

15.

Kuo

Shyu

(2021) A cross-national comparative policy analysis of the blockchain technology between the USA and China. Sustainability 13(12): 6893.

16.

Lester

Neuhoff

(2009) Policy targets: Lessons for effective implementation of climate actions. Climate Policy 9(5): 464–480.

17.

Chao

Kong

, et al. (2020a) The efficiency analysis and spatial implications of health information technology: A regional exploratory study in China. Health Informatics Journal 26(3): 1700–1713.

18.

Wei

Jiang

, et al. (2020b) Equity and efficiency of health care resource allocation in Jiangsu province, China. International Journal for Equity in Health 19: 1–13.

19.

Zhang

Liu

, et al. (2022) Policy evaluation and policy style analysis of ride-hailing in China from the perspective of policy instruments: The Introduction of a TOE three-dimensional framework. Processes 10(10): 2035.

20.

Lin

Yang

Shyua

(2013) A comparison of innovation policy in the smart grid industry across the Pacific: China and the USA. Energy Policy 57: 119–132.

21.

Melnikov

Karelin

(2021) Public procurement as an economic policy tool. Journal of Institutional Studies 13: 100–115.

22.

Qin

Yue

(2020) Evolution of China’s marine ranching policy based on the perspective of policy tools. Marine Policy 117: 103941.

23.

Relman

(2020) To stop the next pandemic, we need to unravel the origins of COVID-19. Proceedings of the National Academy of Sciences 117(47): 29246–29248.

24.

Rothwell

ROY

Zegveld

(1984) An assessment of government innovation policies. Review of Policy Research 3(3-4): 436–444.

25.

Schneider

Ingram

(1990) Behavioral assumptions of policy tools. The Journal of Politics 52(2): 510–529.

26.

Simonofski

Fink

Burnay

(2021) Supporting policy-making with social media and e-participation platforms data: A policy analytics framework. Government Information Quarterly 38(3): 101590.

27.

Tuominen

Himanen

(2007) Assessing the interaction between transport policy targets and policy implementation—a Finnish case study. Transport Policy 14(5): 388–398.

28.

Wang

Zhang

(2022) The impact of policy topic networks on recombinant creation capabilities: Empirical evidence from the energy field. Journal of Cleaner Production 380: 134858.

29.

Wang

Zhao

Zhu

, et al. (2023) Towards intelligent policy analysis: A discourse structure parsing technique for Chinese government document. Information Processing & Management 60(4): 103363.

30.

Wang

(2021) Government policies, national culture and social distancing during the first wave of the COVID-19 pandemic: International evidence. Safety Science 135: 105138.

31.

(2019) “Documentary Politics”: Hypotheses, Process, and Case Studies, Decision-Making in Deng's China. United Kingdom: Taylor & Francis.

32.

Wang

, et al. (2021) Characterizing the patterns of China’s policies against COVID-19: A bibliometric study. Information Processing & Management 58(4): 102562.

33.

Pan

Xin

, et al. (2022) The mediating role of public health between environmental policy tools and economic development. Energies 15(3): 835.

34.

Yang

Huang

(2022) Exploring the diversity and consistency of China’s information technology policy. Journal of Information Science 48: 1–24.

35.

Yang

Huang

(2020) A bibliometrics-based research framework for exploring policy evolution: A case study of China’s information technology policies. Technological Forecasting and Social Change 157: 120116.

36.

Yao

Gong

, et al. (2021) Characteristics and evolution of China’s industry–university–research collaboration to promote the sustainable development: Based on policy text analysis. Sustainability 13(23): 13105.

37.

Zhang

Yao

Evans

, et al. (2022) Collaboration of issuing agencies and topic evolution of health information policies in China. Journal of Information Science 49(6), 1692–1710.

38.

Zhu

Feng

Ning

X, et al.

(2022) Analysis of China's fight against COVID-19 from the perspective of policy tools—policy capacity. Frontiers in Public Health 10: 951941.