Using discrete-event simulation for planning and managing mass vaccination centers: a comprehensive examination

Abstract

Different studies based on discrete-event simulation have been conducted to evaluate the operation of mass vaccination centers and assist healthcare planners in improving their performance. These studies explored different parameters and scenarios for the operation of mass vaccination centers and generated several results. This review aims to assess discrete-event simulation applications in mass vaccination center management. This critical analysis concentrated on every phase of the project method for discrete-event simulation. To ensure comprehensive coverage, scholarly works about the application of discrete-event simulation in mass vaccination centers were sourced from the World Health Organization’s COVID-19 research database and other databases, including Web of Science, Scopus, MDPI, Sage, PubMed, medRxiv, and the WHO COVID-19 research database. Only English-language studies detailing the application of discrete-event simulation in mass vaccination centers were taken into account. The vaccines, simulation software, strategies, or origins were all unrestricted. Hence, the search was expanded to include other vaccination campaigns besides COVID-19. Our investigation included vaccination campaigns against H1N1 and the influenza virus. Eleven studies were selected. The results show that most simulation studies of vaccination centers have several shortcomings that are visible at every phase of the discrete-event simulation project. Through an analysis of the current state of vaccination center simulation studies, our research identifies best practices for upcoming studies. Following these recommendations ought to raise the standard of upcoming studies at large vaccination centers.

Keywords

Discrete-event simulation DES mass vaccination center mass immunization clinic

1. Introduction

First identified in Wuhan, China, on November 16, 2019, the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) virus responsible for COVID-19 quickly spread globally,¹ prompting the World Health Organization (WHO) to declare it a global pandemic on 11 March 2020.^2,3 Despite sustained containment efforts spanning over a year, certain countries struggled to manage COVID-19, while others achieved favorable outcomes. This pandemic, recognized as a significant public health crisis, has not only led to widespread illness and fatalities but has also profoundly impacted every aspect of society, exacerbating the global economic downturn.^4,5 Throughout this period, numerous researchers made notable scientific advancements, resulting in the development of COVID-19 vaccines.⁶ The subsequent challenge was to initiate large-scale vaccination campaigns aimed at inoculating as many individuals as possible against COVID-19 in a timely manner. These campaigns need comprehensive planning and preparation across various fronts. Key aspects of this preparation include raising public awareness, prioritizing population groups for vaccination, managing vaccination appointments effectively, and designing and operating vaccination centers.

To address these challenges, mass vaccination centers (MVCs) have been established and implemented worldwide as pivotal tools in combating the pandemic. MVCs serve as crucial hubs for vaccine administration, offering streamlined processes for vaccination, ensuring fair access to vaccines, and optimizing logistical operations to facilitate efficient vaccine distribution and uptake within communities. In addition, these MVCs play a vital role in addressing public concerns, providing accurate information, and fostering trust in the vaccination process.

Various studies based on discrete-event simulation (DES) have been undertaken to assess the functionality of MVCs and aid healthcare planners in enhancing their effectiveness. These studies have delved into diverse parameters and scenarios concerning MVC operations, yielding a plethora of outcomes. DES stands out as one of the prevailing tools employed for simulating system operations through discrete-event sequences over time, particularly in domains such as manufacturing processes, capacity planning, and investment decision-making.^7–9 Moreover, DES has proven instrumental in optimizing the performance of public health systems, demonstrating its versatility and applicability across multifarious sectors.^10,11

As the global population gradually gains some level of immunity through successive waves of mass vaccination efforts, it is obvious that the COVID-19 pandemic may not be swiftly resolved because of challenges such as limited vaccine supply and concerns regarding vaccine efficacy against emerging variants. In light of these ongoing challenges, it becomes imperative for policymakers and healthcare planners to analyze available data and utilize effective tools for managing MVCs. In this context, a thorough examination of MVC simulation studies becomes paramount to extracting valuable insights and lessons for future pandemics. While previous studies have conducted literature reviews on the application of DES in the healthcare sector, none have specifically delved into the unique dynamics of mass vaccination care, characterized by dense flows of individuals over relatively short periods. Recognizing this gap, our research endeavors to assess the objectivity of studies using DES within the context of mass vaccination through a comprehensive systematic literature review.

At the heart of our investigation lies a fundamental RQ: How has DES been effectively employed in managing operations at MVCs during pandemics? Figure 1 outlines the methodology used to address this question, which involves a systematic review of the literature on DES in MVCs. Selected papers are evaluated qualitatively based on the application of DES project steps. Rather than describing the findings, our focus is on critically examining how DES project steps are implemented in these studies. This critique aims to enhance understanding of how DES can improve mass vaccination response strategies. Our study seeks to provide insights into the optimization of MVC operations, thereby informing future pandemic response strategies and advancing global healthcare preparedness.

Figure 1.

Research methodology.

The next section delves into the background of the DES project method, detailing its principles and applications. The third section outlines the search methodology employed to gather relevant literature for this study. Following this, an overview of the selected studies is presented, highlighting their characteristics and providing detailed insights into the layout of the MVCs analyzed. The fifth section offers a comprehensive critical analysis of these publications, with a focus on evaluating the simulation methodologies used. The sixth section discusses the findings from this analysis. Finally, the concluding section summarizes the key insights and conclusions, providing a concise wrap-up of the research.

2. Background: DES project method

Simulation is a powerful tool for replicating real-world system behavior using dedicated computer software.^12–15 It entails analyzing systems and making operational or resource policy decisions based on the results. This process consists of several steps, beginning with the development of a model of the system, followed by experiments on this model, interpretation of the simulation results, and making informed decisions. Simulation allows for the comparison of different system configurations and the evaluation of various control strategies in order to optimize the system’s performance. Successful simulation projects require planning. Various researchers have identified key stages and their progression in simulation projects, emphasizing the importance of careful planning and execution.^15–17 Figure 2 depicts the general procedure for using DES, including a visual representation of the steps involved.

Figure 2.

DES project.

2.1. Setting objectives

The first stage of the simulation project is critical because it lays the groundwork for the entire effort. The simulation’s motivation is defined, revealing the central problem to be solved and establishing the project’s specific goals and objectives. These objectives are frequently presented as critical questions to be answered or nuanced scenarios to investigate.¹⁸ This phase also includes a detailed definition of the personnel required to ensure the simulation project runs smoothly. The roles and responsibilities of the project team are carefully defined to ensure effective coordination and collaboration throughout the project’s lifecycle. In addition, the study costs and estimated timeframe for completing the simulation project are assessed. This entails an assessment of resource allocation, including software, hardware, and personnel costs.

2.2. Conceptual model

A conceptual model serves as a fundamental representation or abstraction of a system, aiming to convey its essential aspects and relationships.¹⁹ It is not intended to be a detailed or precise replica; instead, it offers a simplified and high-level depiction designed to facilitate understanding, communication, and analysis.¹⁸ In system simulation, a conceptual model outlines the fundamental elements and interactions of a system before being translated into a more detailed simulation model suitable for execution on a computer for analysis and experimentation. During the development of a simulation model, detailed descriptions derived from the conceptual model play a crucial role in guiding the model-building process and coding efforts. These descriptions provide valuable insights into the intricacies of the system, enabling developers to translate the conceptual model into a functional simulation model that effectively captures the system’s behavior and dynamics. By leveraging detailed descriptions derived from the conceptual model, simulation model development becomes more structured and streamlined, ultimately enhancing the accuracy and reliability of the simulated outcomes.

2.3. Data collection and processing

The process of input data modeling is a systematic undertaking that involves various key steps, each critical to ensuring the accuracy and reliability of the simulation model. These steps include data collection, probability assessment, statistical analysis, and subsequent evaluation.²⁰

Various methodologies, such as time study analysis, work sampling, and historical records, are employed for effective data collection. Time study analysis involves observing an operator’s task duration, generating a set of task time observations. Conversely, work sampling determines the percentage of time allocated to different activities, aiding in identifying probabilities associated with task performance and validating simulation model outputs. Historical records also serve as valuable sources for gathering additional data. Once the data are collected, the subsequent step involves modeling it into a statistical distribution, such as normal, exponential, uniform, triangular, Weibull, and so on. Statistical distributions are of particular interest in DES projects because of their ability to replicate the random variations influencing the system’s behavior.

The final stage of the data analysis process entails assessing the adequacy of the modeled statistical distribution for the collected data. This step is crucial for verifying the goodness of fit. Various methods are employed for this purpose, including graphical methods such as histograms, P-P plots, and Q-Q plots,²¹ and statistical tests such as the chi-squared test²² and Kolmogorov–Smirnov Test.²³ These tools enable the evaluation of whether the hypothesized probability distributions accurately match the data, ensuring the reliability of the simulation model.

2.4. Coding the model on the software

Model coding in software is the process of translating a conceptual model into a computer-based model that can be used to generate experimental data. This procedure consists of two essential steps. First, selecting appropriate computer software is critical. Second, the simulation model is created using the selected software. Currently, multiple simulation software programs are available, including but not limited to Arena/Rockwell,²⁴ Simul8,²⁵ AnyLogic,²⁶ Simio,²⁷ and SimPy.²⁸ Each of these platforms offers unique features and capabilities tailored to different simulation needs and preferences. Therefore, selecting the most suitable software is paramount to ensuring the successful implementation of the simulation.

2.5. Model verification

The simulation model verification stage is intricately linked with the simulation model coding phase and holds significant importance in ensuring the accuracy and reliability of the model.²⁹ During this stage, attention is devoted to ensuring that the model encompasses all necessary components and functions correctly.

In model verification, proactive measures are taken to recognize and rectify potential errors. Model debugging plays a pivotal role in identifying and resolving discrepancies in the simulation code. Key errors targeted during this process include issues related to flow control, entity creation, resource release, and logical inaccuracies in observed statistics.

Model debugging activities encompass various strategies, including scenario repetition with identical random number seeds, sensitivity analysis to validate behavioral patterns, and testing individual modules within the simulation code. In addition, a range of tests can be applied for simulation model verification, such as substituting random times with constants, manipulating arrival rates to simulate congestion phenomena, testing specific model segments, and evaluating the model under constrained conditions.³⁰ Another effective verification approach involves the utilization of animation, enabling the visualization and inspection of entity paths and state changes following a predefined wait period.³¹ By employing animation, analysts can gain valuable insights into the model’s behavior and identify any discrepancies or anomalies that require attention.

2.6. Model validation

The validation of the simulation model ensures that it accurately represents the real system. It ensures that the simulation model mimics the behavior of the physical system.^17,31,32 The most effective method for validating the simulation model is to compare its outputs to those of the system using statistical tests.^33,34 Another method for validating the simulation model is to compare the real output value to the confidence interval (CI) of simulation observations. The execution of sensitivity analysis can be used to validate the simulation model, establishing whether alterations in the model’s input data result in predictable changes in the model’s output.

2.7. Experimentation and analysis of results

During the experimentation phase, it is essential to simulate various scenarios and define several parameters for each, including the number of replications, the duration of each replication, and the initial state of the model. The results of these simulations can be modeled by calculating the mean and variance or by establishing CIs. Depending on the objective of the analysis, statistical tests may also be applied.^17,35,36 To compare two scenarios or models, hypothesis testing or CIs can be used.^17,35,36,37 When comparing three or more scenarios or models, a two-step approach involving analysis of variance (ANOVA) and Duncan’s multiple range tests, is applied.^17,35,36,38

3. Search method

Using the PRISMA guidelines for writing reports, we created a systematic review.³⁹ Figure 3 summarizes the search method and the final number of articles included. This approach ensured that only the most relevant and pertinent articles were reviewed, thereby enhancing the quality and reliability of the analysis.

Figure 3.

Flow chart of this review.

3.1. Strategy

In February 2024, a comprehensive and systematic search was conducted across various academic and scientific databases to identify relevant literature on the application of DES in MVCs. The databases searched included Web of Science, Scopus, MDPI, Sage, PubMed, medRxiv, and WHO COVID-19. No restrictions were placed on publication date to ensure the inclusion of both recent advancements and foundational studies. This approach aimed to encompass a broad spectrum of relevant literature, including peer-reviewed journal articles.

3.2. Inclusion criteria

The focus of the selected studies had to be on the modeling of MVCs. To ensure that the studies were relevant, they had to contain specific key terms in their titles, abstracts, or key terms. These terms included “discrete-event,” “discrete event,” “simulation,” “DES,” “centre,” “clinic,” “mass,” “vaccination,” and “immunization.” The presence of these terms indicated that the studies were likely to involve the types of methodologies and topics pertinent to the research question.

Moreover, the scope of the included studies was broad, allowing for the inclusion of any research related to mass vaccination, not just those focused on COVID-19 vaccines. This decision was made to ensure that valuable insights from earlier vaccination campaigns, which might involve different vaccines, could be integrated into the analysis. Such insights could offer lessons and strategies applicable to current and future mass vaccination efforts. In addition to their thematic focus, the articles were required to provide detailed information on the planning, organization, and execution of an MVC. This criterion was crucial because the review aimed to gather practical insights that could be applied in real-world settings. The inclusion of such information ensured that the studies were not just theoretical but also offered actionable knowledge.

Finally, the studies needed to offer practical insights into the use of DES in the context of MVCs. DES is a powerful tool for modeling and optimizing complex processes, and its application in MVCs can lead to significant improvements in efficiency and effectiveness. By focusing on studies that utilized DES, the review could draw on the best available evidence for how these simulations can be used to enhance the planning and operation of MVCs.

3.3. Exclusion criteria

Articles written in languages other than English were excluded to ensure consistency in language and comprehension, as well as to avoid potential misinterpretations that could arise from translation. This decision also helped streamline the review process, allowing the researchers to focus on a consistent body of literature without the complications of language barriers.

Moreover, studies that did not specifically focus on the modeling of MVCs were excluded because the primary aim of the review was to gather insights into how MVCs can be effectively planned, organized, and executed. Including studies with different focuses would have diluted the relevance of the findings and made it harder to draw clear conclusions applicable to MVCs.

Finally, any studies unrelated to mass vaccination were excluded from the review. The focus of the research was on vaccination efforts, specifically the modeling and operation of MVCs, and studies outside of this scope would not provide the necessary insights or data. By applying these exclusion criteria, the review ensured that only the most relevant, practical, and focused studies were included, enhancing the overall quality and applicability of the findings.

3.4. Search results

The initial literature search yielded a total of 66 articles. However, this figure did not represent a fully exhaustive count, as many studies were retrieved from multiple databases, leading to instances of duplication. To ensure the accuracy and integrity of the analysis, these duplicates needed to be carefully identified and removed. A systematic process of cross-referencing was employed, during which 15 duplicate articles were identified and subsequently eliminated from the pool. This step reduced the dataset to 51 unique articles, each representing distinct contributions to the field.

With a refined set of articles, the next phase involved assessing their relevance to the study’s objectives. This was initially done through a thorough examination of the titles and abstracts of each article. The goal of this screening was to quickly identify studies that closely aligned with the focus of the review. Through this process, 36 articles were found to be insufficiently relevant, either because their focus diverged from the core topic of modeling MVCs or because they lacked the necessary depth or context. These articles were excluded, narrowing the field to 15 potentially valuable studies.

The remaining 15 articles were then subjected to a more rigorous and detailed review. This second screening involved a closer examination of each article’s methodology, findings, and overall contribution to the subject. The aim was to determine whether these studies provided the practical insights and data necessary to meet the study’s inclusion criteria. During this in-depth evaluation, four additional articles were identified as irrelevant. These articles were excluded due to factors such as a lack of direct applicability to MVCs, insufficient methodological rigor, or a focus that was too tangential to the study’s core objectives.

Finally, after this comprehensive two-step screening process, 11 articles were identified as meeting all the inclusion criteria. These selected articles were deemed to be of high relevance and quality, offering valuable insights and practical guidance on the modeling and implementation of MVCs. These studies formed the foundation of the final review, ensuring that the analysis was grounded in the most pertinent and reliable evidence available.

4. Overview of studies

4.1. Studies characteristics

From 2009 to 2024, 11 papers were published on DES of MVCs, showing a consistent annual growth rate of 5.08%. The initial article emerged in 2009 during the influenza A(H1N1) outbreak, and two more were published in 2013 and 2014, corresponding to the start of the A(H3N2) influenza epidemic. With the release of COVID-19 vaccines in 2020, studies increased, peaking with three articles in 2021 and 2023. These publications appeared in 10 sources, including 9 scientific journals and 1 international conference. “Healthcare” journal published two articles. The journals’ impact factors ranged from 1.6 to 7.8, indicating their good stature.

Thirty-two authors contributed to these studies, with Asgary A. being the most prolific author, presenting three distinct studies. Examining Scopus citation counts, Asgary et al.⁴⁰ leads with 52 citations, followed by Gupta et al.,⁴¹ Pilati et al.,⁴² Washington,⁴³ Beeler et al.⁴⁴ and Asgary et al.Jerbi and Masmoudi⁴⁶ and Sala et al.⁴⁷ studies have one citation each. Adjusting for the annual citation rate places Asgary et al.⁴⁰ at the top with 13 citations per year, followed by Pilati et al.,⁴² Gupta et al.,⁴¹ Asgary et al.,⁴⁵ Beeler et al.⁴⁴ and Washington⁴³ with annual rates of 6.33, 3.09, 2.67, 1.3, and 1.13, respectively.

This number of citations is a significant indicator for several reasons: it reflects the impact and influence of an author’s work within the academic community, with highly cited authors often recognized as key contributors in their field, indicating that their research is foundational or highly relevant. Although not the sole measure of quality, a higher citation count suggests that the work is considered rigorous, reliable, and valuable, often implying that it has been peer-reviewed and has withstood the test of time. Citations also highlight the relevance of an author’s work to the current state of research, showing that frequently cited works address important questions or offer methodologies widely applicable across the field. In bibliographic reviews, highly cited works can be identified as key contributions that have shaped the direction of research, playing a central role in the development of theories, methods, or practices. In addition, citation counts guide researchers toward the most influential and potentially reliable sources, helping them prioritize reading and identify impactful studies to build upon. Over time, analyzing citation trends can provide insights into the evolution of a discipline, revealing which topics or methods have gained prominence or lost influence. Finally, citation numbers serve as a benchmark for comparisons, allowing for an evaluation of the relative importance of various contributions within a topic.

4.2. Vaccination center layout

All articles, except for Angelopoulou and Paul⁴⁸ and Asgary et al.,⁴⁰ reported real-case applications (Table 1). Seven articles have been implemented in the United States and Canada; two have been implemented in Italy, one in Tunisia, and another in China. In addition, eight articles were on COVID-19 vaccination^{40,42,45–50} two were on influenza vaccination,^43,44 and only one was on H1N1 vaccination (Table 1).⁴¹ The studies covered various vaccination center types, with seven articles assessing walk-in MVCs^43,44,46,47 and the remaining articles evaluating drive-through MVCs.^{40–42,45,48} Notably, Asgary et al.⁴⁹ introduced a novel type of walk-in MVC where patients are accommodated in private boxes for the duration of the vaccination procedures, with health professionals administering vaccinations while moving between boxes (Table 1).

Table 1.

MVCs DES studies characteristics and MVC layout.

Study	Studies characteristics			Vaccination center characteristics
	Country	Vaccine	Study type	MVC type	MVC location
Asgary et al.⁴⁰	Canada	COVID-19	Not a real case	Drive-through	A parking lot
Gupta et al.⁴¹	Louisville, USA	H1N1	Real case	Drive-through	A stadium
Pilati et al.⁴²	South Tyrol, Italy	COVID-19	Real case	Walk-in	A baseball stadium parking lot
Washington⁴³	Henderson, USA	Influenza and Pneumococcal	Real case	Walk-in	A Vacant store
Beeler et al.⁴⁴	Ontario, Canada	Influenza	Real case	Walk-in	Not reported
Asgary et al.⁴⁵	Denver, Colorado, USA	COVID-19	Real case	Drive-through	Vaccination clinic
Jerbi and Masmoudi⁴⁶	Sfax, Tunisia	COVID-19	Real case	Walk-in	Not reported
Sala et al.⁴⁷	Bergamo, Italy	COVID-19	Real case	Walk-in	A university sports center
Angelopoulou and Paul⁴⁸	USA	COVID-19	Not a real case	Drive-through	Not reported
Asgary et al.⁴⁹	Ottawa, Canada	COVID-19	Real case	Walk-in^a	A field hockey stadium
Wang et al.⁵⁰	Huzhou and Shanghai, China	COVID-19	Real case	Walk-in	Two gymnasiums and seven medical facilities

Health professionals administer vaccinations as they move between boxes.

Two critical factors influencing the size and location of MVCs are space availability and ease of access. According to Lee et al.,⁵¹ a large area is necessary to accommodate a maximum number of citizens while maintaining adequate distance. Furthermore, MVCs, especially drive-in ones, should be strategically located near major roads, highways, or expressways to facilitate multiple lines of traffic and prevent traffic congestion. Most studies housed their MVCs in non-medical care buildings, with some utilizing vacant stores⁴³ or parking lots as alternative locations.^40,45 Sports facilities were also utilized in certain cases.^41,47,49,50 However, only Pilati et al.⁴² and Wang et al.⁵⁰ opted to locate their MVCs in medical facilities. Interestingly, some authors did not specify the exact location of their MVCs (Table 1).^42,44,46,48

Several activities are required during the vaccination process. Depending on the activity, a volunteer or a healthcare provider may only perform it. Either of them can also perform it with no constraints. The selected articles consider the following activities (Figure 4):

Screening: Upon arrival at the MVC, patients are directed through an organized checkpoint to ensure a smooth flow of traffic and efficient verification of reservations, a process managed either by a single operator or a coordinated team.^46,47 This initial checkpoint serves as the frontline defense against potential health risks, as operators diligently conduct temperature checks on each patient, swiftly identifying any signs of fever or other contagious symptoms, thereby upholding stringent safety measures.²⁹ Following this, patients undergo a brief yet essential disinfection process, maintaining hygienic standards within the MVC premises. Furthermore, operators inspect each patient to verify the proper fitting of their facial mask, ensuring adherence to safety protocols and minimizing the risk of viral transmission.^44,46,47 It is worth noting that this critical screening task is typically entrusted to dedicated volunteers, highlighting the collaborative efforts and community engagement integral to the efficient functioning of the MVC.⁴⁶

Registration: Following the screening process, patients proceed to the registration area, where attention is given to verifying their identity and gathering essential demographic information, essential for maintaining an accurate record and ensuring seamless vaccine administration.^44,47 In addition, patients are provided with comprehensive informational materials about the vaccine, empowering them with the necessary knowledge and ensuring informed decision-making regarding their healthcare.^44,47 Patients may also receive consent forms, enabling them to provide informed consent for vaccination.^{40–42,45,48} This operation is solely managed by dedicated volunteers, reflecting the pivotal role of community engagement in facilitating the registration process and fostering a supportive environment within the MVC.

Medical assessment: This pivotal stage of the vaccination process prioritizes patient safety and informed decision-making. Patients undergo a thorough medical assessment, starting with detailed inquiries into any known allergies or past adverse reactions to vaccines, ensuring a comprehensive understanding of their medical history and potential risk factors. Subsequently, a qualified healthcare professional, often a doctor, conducts an examination of the patient, evaluating their current health status and assessing their suitability for vaccination based on individual risk factors and medical considerations.^42,46,47 This personalized approach underscores the commitment to patient-centered care and the prioritization of safety within the MVC, ensuring that vaccination decisions are made with the utmost consideration for each patient’s unique health needs and circumstances.

Vaccination: Once patients have successfully completed the preceding stages and meet all health requirements, they proceed to the pivotal vaccination stage, marking a significant milestone in their journey toward vaccination.^{40–42,44–48} Skilled healthcare professionals, typically nurses, carry out this essential task with precision and care, administering the vaccine to each patient according to established protocols and guidelines. Notably, certain studies have integrated vaccine resupply and preparation seamlessly into the vaccination phase, streamlining operations and optimizing efficiency within the MVC.⁴⁶ However, it is important to highlight that vaccine administration may be temporarily paused in the event of an awaiting vaccine resupply; ensuring that the process remains uninterrupted and patients receive their vaccinations in a timely manner. This strategic approach underscores the commitment to effective resource management and the prioritization of patient safety and satisfaction within the MVC.

Validation: Following the vaccination process, a crucial step known as validation is undertaken to ensure accurate and up-to-date recording of vaccination status within the information system’s database.⁴⁶ This operation plays a pivotal role in maintaining comprehensive and reliable records of vaccinated individuals, facilitating effective monitoring and management of vaccination efforts. Each vaccinated individual is required to undergo this validation procedure, which involves updating their vaccination status in the database. As part of this process, individuals receive a vaccination SMS, which serves as official documentation of their vaccination status. To streamline this operation, dedicated volunteers utilize a specialized computer application accessible via the Internet, enabling efficient and secure management of vaccination data.

Recovery Area: Following the administration of their vaccination, patients are directed to the designated recovery area, where they are carefully monitored for a duration of 10–30 minutes to assess for any potential adverse effects or reactions to the vaccine.^{40,42,44,46,47} This critical step in the vaccination process prioritizes patient safety and well-being, ensuring prompt identification and management of any adverse reactions that may arise. During this observation period, patients receive attentive care and support from healthcare professionals, including doctors, who are readily available to provide assistance and medical intervention if required. Patients are encouraged to remain within the recovery area until the designated observation period is complete. If no adverse effects are observed and the patient is deemed medically stable, they are then cleared to exit the system, allowing for a seamless and efficient flow of individuals through the MVC.

Figure 4.

Typical process in a mass vaccination center.

Upon arrival at the MVC, individuals undergo screening and registration procedures at a centralized service station, where their demographic information is collected and initial assessments are conducted. Following this initial phase, individuals may be directed to different lanes or designated areas based on their specific needs or requirements for further evaluation and vaccination procedures (Figure 4).

To manage patient flow effectively, MVCs may incorporate multiple waiting areas strategically positioned throughout the facility.^40,43,44,50 The first waiting area is typically located near the entrance of the MVC, providing individuals with a comfortable space to wait before undergoing initial screening and registration processes.³⁷ Beeler et al.⁴⁴ designed this area with consideration for patient comfort and convenience, as prolonged wait times may deter individuals from proceeding with the vaccination process. In addition, a second waiting area may be situated between the screening and registration sections, allowing individuals to transition smoothly between these stages while maintaining an organized and efficient workflow.^{40,45–47,50} This intermediate waiting area serves as a buffer zone, facilitating the seamless movement of individuals through the MVC. Furthermore, a final waiting area may be designated to separate the medical evaluation and vaccination zones, providing individuals with a comfortable space to wait before receiving their vaccination. This area ensures that individuals have adequate time to undergo medical assessments and consultations before proceeding to the vaccination stage, contributing to the overall efficiency and effectiveness of the MVC (Figure 4).

Overall, the strategic layout of MVCs, including the incorporation of designated waiting areas, plays a crucial role in optimizing patient flow, minimizing wait times, and enhancing the overall patient experience during the vaccination process, ultimately contributing to the success of mass immunization efforts (Figure 4).^40,43,44,50

5. Studies analysis

In evaluating the effectiveness of integrating this phase into a DES project, numerous considerations come to light. Table 2 succinctly outlines these considerations for every stage of the simulation project.

Table 2.

Checklist for simulation project stages.

Stage	Questions	Responses
Setting objectives^15,14,18	– Are the study objectives clearly stated?	– Scope, decision problem and modeling objective
	– Are the personnel responsible for each simulation phase listed, along with the associated costs and deadlines?	– Simulation stakeholders: experts, analyst or decision maker, etc– Simulation project costs– Simulation project deadlines
Conceptual model^14,15,18,19	– Is a MVC conceptual model provided?	– Components of the MVC: emplacement, stages, personal, etc– Data and times of the MVC– Flows in the MVC
	– Is the conceptual model provided sufficiently detailed?	– Simplicity in the model is desirable for ease of analysis, validation, and description. However, the model must be sufficiently detailed to ensure that all components, interactions, processes, and flows are accurately represented.
Data collection and processing^14,15,32	– What method is used to collect input data?	– How input data are collected– Number of collections– Period of collection
	– What characteristics define the nature of the data?	– All modeling studies should include an assessment of data uncertainty as it pertains to the decision problem being addressed:– Stochastic,– Determinist
	– Are the data modeled as statistical distributions?	– Nature of statistical distribution and characteristics: mean, variance, etc
	– If so, what statistic tool is involved?	– Tool used to model this distribution to judge it’s accuracy
	– Is the goodness of the data fit to the statistical distribution verified?	– The accuracy of the distribution goodness is verified or not: a p-value
	– If so, what statistical tool is involved?	– Tool is used to verify this accuracy
Coding the model on the software^14,15,17	– What tool is used for coding their simulation model?	– The software used to code the model: Arena, AnyLogic, Simul8, etc
Model verification^14,15,31	– Is the simulation model verified?– If so, what method is used?	– Describe the method used to verify the model: animation, comparison, etc
Model validation^14,15,31	– Is the simulation model validated?– If so, what method is used?	– Describe the method used to validate the model: statistical comparison, ANOVA, CI, etc– Evaluation of face validity of the structure, evidence, problem formulation, and results of the model
Experimentation and analysis of results^14,15,52,53	– What method is employed to explore the developed simulation model?	– Describe the method used to explore the model: design of experiments, what if scenarios, etc
	– How many replications are adopted in the exploration process?– What is the duration of each replication?	– Set up of the simulation: number of replications, durations of replications, warm up period, etc
	– Which metrics are utilized to gather the simulation outputs?– In what format have these metrics been expressed?	– Definition of the outputs– Format of the outputs: simple value, mean, confidence interval, etc
	– What method is employed to explore the results of the developed simulation model?	– Describe the method used to explore the results: simple comparison, sensitivity analyses, ANOVA, etc

To comprehensively gauge the quality of execution during this phase, it is essential to delve into various aspects. Attention should be given to the accuracy and relevance of the data inputs utilized throughout the simulation. Are the data sources reliable, and do they adequately represent the real-world scenario being simulated? In addition, the modeling assumptions made during this phase need to be critically evaluated. Are these assumptions realistic, and do they align with the objectives of the simulation study? Furthermore, the robustness of the model verification and validation processes warrants careful scrutiny. Has the model been thoroughly tested and validated against real-world data or empirical evidence? Moreover, the extent to which the simulation results align with expected outcomes and real-world observations should be thoroughly assessed. Do the simulation outputs accurately reflect the anticipated performance of the system being modeled?

By systematically addressing these questions at each stage of the simulation project, researchers can ensure a comprehensive evaluation of the quality of implementation of this crucial phase in DES. This holistic approach not only enhances the reliability and validity of the simulation outcomes but also fosters greater confidence in the insights derived from the simulation study.

5.1. Setting objectives

DES studies in MVCs encompass a diverse array of objectives. These objectives include evaluating MVC performance in relation to the influx of arrivals,^43,46 estimating staffing requirements for MVCs, determining the impact of changes in staffing levels on MVC performance,^41,44,48,50 and optimizing MVC layouts by modifying their structures, comparing different layouts, estimating the number of stations, or combining two activities.^40,41,45,48 This diversity of objectives underscores the broad spectrum of research goals, encompassing various facets of vaccination center management. These goals span staffing, the layout of vaccination operations, and how MVCs respond to variability in the arrival of individuals to be vaccinated. Several studies have explored additional objectives. Beeler et al.⁴⁴ and Angelopoulou and Paul,⁴⁸ for example, investigated various approaches for prioritizing vaccinations or establishing eligibility restrictions. Pilati et al.⁴² created a digital twin for real-time administration and MVC design, while Asgary et al.⁴⁹ investigated high-capacity facilities for implementing MVCs (Supplemental Table A1).

We observed that the majority of the selected studies did not disclose information about the initiators of the projects. Moreover, while costs may be considered secondary during a health crisis, the absence of a thorough cost analysis could lead to significant financial implications. This critique becomes more relevant as it applies to all the cases examined. Furthermore, none of the studies outlined the duration required to complete MVC simulation projects. Neglecting this temporal aspect may pose a risk to the overall success of MVC simulation initiatives, particularly in times of health crises where swift project completion is essential to enhancing MVC responsiveness (Supplemental Table A1).

5.2. Conceptual model

The main shortcoming is the absence of solid conceptual models of MVCs. Indeed, only Pilati et al.⁴² and Sala et al.⁴⁷ present solid conceptual models of their MVCs. This disparity could have a significant impact on the simulation results’ quality. The absence of a solid conceptual model can lead to arbitrary parameter choices, oversimplification, and conceptual inconsistencies. As a result, simulation results may lack credibility and validity, making it difficult to draw conclusions and limit confidence in the recommendations made. Some researchers employ overly simplified conceptual models that are incapable of capturing the true complexities of MVCs.⁵⁰ Oversimplified conceptual models may underestimate the interactions and interdependencies between the various components of the MVC, resulting in inaccurate simulation results. By ignoring critical aspects of operational reality, these limited conceptual models can provide ineffective recommendations for MVC management.

5.3. Data collection and processing

In most studies, data were gathered from real MVCs and used to estimate the service time values for vaccination operations.^{41,43–46,49} Pilati et al.⁴² have developed a mobile application to instantly collect service times. Asgary et al.⁴⁰ combined some theoretical times they had selected with additional times determined by other studies. Angelopoulou and Paul⁴⁸ adopted vaccination processing times from past simulation studies (Supplemental Table A2).

The service times were stochastic in six studies^{43,44,46–48,50} and constant in two other studies.^45,49 Asgary et al.⁴⁰ combined stochastic and constant times in their investigation. Because constant service times ignore common real-world variations in systems, they can significantly affect simulation results. Therefore, the simulation results will be overly stable; making them unrepresentative of the actual dynamics of MVCs. Decisions based on such simulations may thus be ineffective because they do not account for the variation factors that exist in the actual vaccination center. The lack of a description of the service times by Gupta et al.⁴¹ made it difficult to verify the results and reproduce the simulations. The transparency of simulation model data is intended to allow other researchers to replicate experiments, conduct independent analyses, and build confidence in the results. When specific values are unavailable, researchers’ ability to validate and verify results is limited, making conclusions less convincing (Supplemental Table A2).

Only Washington,⁴³ Jerbi and Masmoudi⁴⁶ and Sala et al.⁴⁷ used tools to align MVC service times with statistical distributions. In addition, Washington⁴³ and Sala et al.⁴⁷ used chi-square and Kolmogorov–Smirnov statistical tests to assess how well service time data fit statistical distributions. However, Jerbi and Masmoudi⁴⁶ do not employ such tests. The adequacy of the fit of the data to the probabilistic distribution must be verified to ensure the validity of the results. Without such verification, the results may be sensitive to adjustment errors, leading to incorrect conclusions (Supplemental Table A2).

5.4. Coding the model on the software

Arena/Rockwell is the most commonly used software in MVCs’ DES. This software is used in the simulation model coding phases of five studies.^{41,43,46,48,50} AnyLogic was used to create the simulation models for the three studies by Asgary et al.⁴⁹ and Asgary et al.⁴⁰ Beeler et al.⁴⁴ and Sala et al.⁴⁷ used Simul8 and FlexSim, respectively. Only Pilati et al.⁴² were vague about the software they used to create their simulation model (Supplemental Table A2).

5.5. Model verification

None of the MVC DES studies described the verification stage except for Sala et al.⁴⁷ In this study, the simulation model was shown to health professionals for verification. When simulation model verification is not performed, coding errors can occur, affecting the results. Indeed, verification issues can lead to unreliable results and biased interpretations because the model does not operate according to the intended specifications. In this context, verification errors can lead to a poor assessment of MVC performance and bad decision-making (Supplemental Table A2).

5.6. Model validation

Information on the simulation models’ validation was absent from six studies.^{40,42,45,48–50} Washington,⁴³ Gupta et al.⁴¹ and Beeler et al.⁴⁴ used a simple comparison between the simulation results and real-world center results to validate their simulation models. Jerbi and Masmoudi⁴⁶ and Sala et al.⁴⁷ respectively, performed a t-test and an ANOVA statistical test to compare the simulation models and real center results. Validation is critical for determining whether the model accurately represents reality. When this step is skipped, simulation results may not be applicable to real-world scenarios. The model may not accurately represent the real system’s mechanisms and behaviors. As the simulation results are not based on a validated simulation model, this can lead to incorrect decisions (Supplemental Table A2).

5.7. Experimentation and analysis of results

The 11 simulation studies of MVCs used one of three exploration techniques. The “what-if” scenario is the most commonly used one, in which the authors consider various center designs or various arrivals.^{41,43,45–50} The second technique is sensitivity analysis, which looks at how the center reacts when one of its characteristics is varied.^40,44,45,50 These characteristics can be structural, like the number of stations and vaccination lines, or related to the citizen, like their inter-arrival times, their patience with waiting, and the disease transmission rate. The last technique is the full factorial design of experiments based on the variation of the MVC characteristics.^41,42,44 These elements typically include the service times, number of employees, length of waiting lines, or priority restrictions (Supplemental Table A3).

During this phase, all the cases examined specified the length of each simulation for each replication. The simulation duration is critical because it determines how long the model will run for each scenario. When this information is missing, determining the precise temporal scope of the simulation experiment becomes difficult, which can lead to difficult-to-compare or generalize conclusions. Because the duration of the simulation frequently influences the dynamics and trends observed, the results can be incomplete or uninformative. Similarly, the number of replications performed per simulation is very important. Only one case study fails to specify the number of simulation replications.⁵⁰ The number of replications, or iterations, directly affects the statistical accuracy of the simulation results. When this parameter is not specified, it becomes difficult to assess the reliability of the results. Inadequate replication numbers can lead to high variability in the results, making it difficult to detect significant trends or make sound decisions. Without this information, it is difficult to determine the credibility of the simulation’s findings. Many studies, on the other hand, have built their experiments around a single simulation replication.^40,42,45,49 This eliminates one of DES’s primary benefits, the model’s stochasticity. Many systems, as for any MVC, are inherently uncertain and subject to variability, making them better suited for stochastic modeling rather than deterministic approaches. A stochastic DES model accounts for this inherent randomness by incorporating probability distributions to represent the various inputs and processes within the system. As a result, each replication of the model can yield different outcomes, reflecting the natural variability and unpredictability of real-world systems. This variability in results from replication to replication is not a drawback but rather strength of stochastic models. It allows for a more nuanced and accurate representation of the system being studied. By capturing the range of possible outcomes, stochastic models provide a deeper understanding of the system’s behavior under different conditions, offering insights that would be missed by a deterministic model that assumes fixed inputs and processes. Moreover, stochastic models can help identify the likelihood of various scenarios, assess risks, and inform decision-making by highlighting the range of possible outcomes rather than a single, deterministic prediction. This approach is particularly valuable in complex systems where uncertainty plays a significant role, such as in operations research, epidemiology, finance, and logistics. By embracing the variability inherent in the system, stochastic modeling provides a more realistic and comprehensive view, leading to more informed and robust decisions (Supplemental Table A3).^17,32,35

The majority of studies analyzed experimental results using only simple comparisons or sensitivity analyses.^40,44,47 Other studies use statistical analysis tools such as the student t-test, ANOVA,³¹ and Arena/Process Analyzer.^41,48 The last part of these studies used the OptQuest optimization tool, which is embedded in the Arena simulation software.^41,43,50 The simulation results were analyzed using a variety of performance measurement categories. The first category concerns the number of vaccinations administered in MVCs.^47,49,50 This number was measured in different ways, including throughput, vaccinations per nurse, the total number of vaccinations, the total number of vaccinations administered on time, and the total number of vaccinations administered. The second category includes both waiting time and system time.^{40,41,43–49} The third category consists of operator and vaccination site utilization rates.^42,46,47 Washington⁴³ and Beeler et al⁴⁴ used additional performance measures such as expected infections in the center, vaccination costs, and center revenue. Gupta et al⁴¹ and Jerbi and Masmoudi⁴⁶ present their performance measurements as CIs, while all other studies present them as mean values.

The lack of statistical analysis of the results, which is present in six of the studies reviewed,^{40,42,44,46,47,49} is the major shortcoming in this phase of the simulation project. Statistical analysis is required to assess the significance and dependability of simulation results. It is difficult to distinguish between random variations and significant trends without it. This means that vaccination center management decisions can be made based on uncertain outcomes, increasing the risk of poor decisions (Supplemental Table A4).

6. Discussion

Simulation studies of MVCs are often sporadic, primarily emerging as reactive measures during pandemic outbreaks to address the urgent need for mass vaccination strategies. This immediacy compresses research efforts into short timeframes, resulting in bursts of activity that fade once the crisis subsides. Such a reactive approach contributes to a lack of continuity and consistency in the literature, leaving few studies available for comparison or as foundations for future research.

Most existing MVC simulation studies focus on the COVID-19 vaccination campaign, reflecting the pandemic’s global scale and urgent demand for effective vaccination strategies. These studies have been essential in helping policymakers and public health officials plan efficient vaccine administration for large populations under diverse conditions. However, this focus on COVID-19 has led to relatively limited research on other vaccination campaigns, such as those for influenza, polio, or measles. These earlier studies, though valuable, are fewer and often lack the extensive data available during the COVID-19 pandemic, resulting in a limited body of work examining MVC strategies across different diseases and contexts.

The scarcity of comparable studies across different vaccination campaigns presents challenges for researchers and public health officials seeking to develop generalized models or draw lessons for future pandemics and routine mass vaccination programs. While the focus on COVID-19 has advanced MVC simulation, it highlights the need for a more systematic and proactive approach to studying mass vaccination strategies. Broadening MVC research to encompass a wider range of diseases and contexts would create a more versatile body of knowledge applicable to various public health challenges. Furthermore, the sporadic nature of MVC simulation research underscores the importance of continuous research, even outside of pandemic crises. Investing in adaptable models and sustained research efforts would help ensure that insights from past experiences contribute to an evolving knowledge base, strengthening future public health strategies.

Geographically, most existing studies have been conducted in the Americas and Europe, with fewer focusing on Asia and Africa. This concentration has led to models that are tailored to the healthcare infrastructure and logistical challenges specific to these regions, potentially limiting their applicability to areas with different characteristics. To create a universally applicable simulation model, it is essential to integrate data reflecting diverse healthcare settings and logistical conditions across regions, including population density, healthcare infrastructure, vaccination center capacities, and supply chain logistics. These data will help the model adapt to varying regional conditions. Proven elements from previous studies, such as DES principles for managing patient flow, queue management, staff optimization, and efficient site structuring, should be retained. However, model parameters like arrival rates, service times, and resource availability should be adjustable to reflect local variations. By experimenting with variables such as vaccine availability, demographics, and public health policies, the model can offer tailored insights for different settings. For regions with unique challenges, in-depth studies can guide adjustments, prioritizing resource allocation and minimizing travel time in underserved areas, or managing high patient volumes and preventing overcrowding in dense urban areas.

Most MVC simulation studies are anchored in real-life scenarios, providing insights into patient flow, resource constraints, and logistical challenges encountered in actual vaccination campaigns. This real-world basis makes the findings highly relevant to public health planning and emergency response. A subset of studies, however, employs hypothetical scenarios, which allow researchers to explore “what-if” situations and prepare for potential future events, such as varying vaccine availability, diverse population densities, or new infectious diseases. These hypothetical models are valuable for testing different strategies and developing contingency plans for future needs.^{41,43,45–50}

The simulation studies reviewed cover various MVC formats, including walk-in and drive-in centers. Walk-in MVCs, common in densely populated urban areas with robust public transportation, focus on managing patient flow in confined spaces, optimizing layout design, staffing, and queue management.^{42–44,46,47,49,50} Drive-through MVCs, typically implemented in rural or suburban areas where individuals remain in their vehicles, emphasize managing vehicle flow and coordinating resources to prevent bottlenecks.^41,45,48 By analyzing both formats, these studies offer insights that help public health authorities tailor vaccination strategies to the unique needs of urban and rural areas, enhancing the effectiveness of mass vaccination campaigns across diverse regions.

Several significant gaps were identified in the reviewed MVC DES studies, particularly in reproducing these studies and evaluating their overall quality. These gaps hinder the ability to compare findings across studies and prevent future research from building on existing work. The diversity of approaches and methodologies, while addressing specific local conditions, also complicates generalization and the application of lessons across contexts. As a result, these challenges impede the development of a solid, cumulative body of knowledge in the field. The identified gaps can be categorized into three main areas, as outlined in Table 3:

Failure to report key simulation project phases: This gap is particularly evident across several phases, including conceptual model construction, data collection and processing, model verification and validation, and results analysis. The absence of a clear conceptual model, which outlines assumptions, parameters, and relationships, limits transparency and hinders replication. Similarly, insufficient reporting on data sources, assumptions, and pre-processing steps obstructs the evaluation of input data quality, potentially compromising results. Furthermore, many studies omit model verification and validation, raising concerns about the accuracy of findings. Without these steps, it is difficult to confirm whether the simulation accurately reflects real-world dynamics. Finally, the lack of detail on result analysis, such as statistical methods or sensitivity analysis, limits the ability to assess the robustness and significance of the findings. This category represents the majority of gaps in MVC simulation studies, with nearly all studies showing some inadequacies, and one study affected by all of these gaps.⁴⁹

Lack of information: This gap presents significant challenges for reproducibility and quality assessment in MVC DES studies. Missing details, such as the location of vaccination centers, hinder understanding of the specific context in which the simulation was conducted.^40,41,49 The physical location can influence factors like population density, accessibility, and demand fluctuations, all of which are essential for accurate results. Without this information, replicating the study in comparable contexts or adapting it to different settings becomes difficult. In addition, the absence of data on the simulation tool used and service times at centers complicates reproducibility.⁴² Without knowing the simulation tool or service times, cross-study comparisons and reliable assessments of the vaccination process are limited.⁴¹ Moreover, missing contextual data, such as center capacity, staff availability, or surrounding community demographics, affects the ability to critically evaluate the relevance and applicability of findings. Ultimately, this lack of information hampers replication, impedes future research, and restricts the development of a cumulative knowledge base in the field.

Improper data parameterization: The assumption of constant service times in MVC DES studies presents significant challenges for reproducibility and quality assessment.^40,45,49 By neglecting the variability in service times, caused by factors like patient arrival rates, staff availability, and procedural complexities, these studies fail to reflect real-world dynamics, where conditions are rarely static. This unrealistic assumption makes it difficult for other researchers to replicate studies under comparable conditions, as it misrepresents actual MVC operations. Furthermore, the lack of variability in service times undermines the study’s applicability to real-world settings, where fluctuations in demand and patient needs are common. This gap also complicates comparisons with studies using more realistic data assumptions, isolating these studies and weakening their contribution to the field. In terms of quality assessment, constant service times can lead to misleading conclusions about capacity, waiting times, and resource needs, affecting public health planning and decision-making. Without incorporating realistic variability, these studies lack robustness and fail to provide reliable insights for diverse MVC contexts.

Table 3.

MVCs DES studies gaps.

Gaps	Simulation project phase	Nature	Gravity	Asgary et al.⁴⁰	Gupta et al.⁴¹	Pilati et al.⁴²	Washington⁴³	Beeler et al.⁴⁴	Asgary et al.⁴⁵	Jerbi and Masmoudi ⁴⁶	Sala et al.⁴⁷	Angelopoulou and Paul⁴⁸	Asgary et al.⁴⁹	Wang et al.⁵⁰
Failure to report one simulation project phase	Conceptual model	Conceptual model not reported	1	X	X		X	X	X	X		X	X
	Data collection and analyze	Adjustment of service time data not reported	5	X	X			X	X			X	X	X
		Accuracy of the service time adjustment not reported	5	X	X	X		X	X	X		X	X	X
	Verification	Verification of the simulation model not reported	2	X	X	X	X	X	X	X		X	X	X
	Validation	Validation of the simulation model not reported	3	X		X			X			X	X	X
	Experimentation	Output statistical analyses not reported	5			X					X		X
Lack of information	Conceptual model	Center location not reported	2	X	X								X
	Data collection and analyze	Times service not reported	4		X
	Simulation model coding	Simulation tool not reported	1			X
Improper data parameterization	Data collection and analyze	Constant service times	5	X					X				X
Total of frequency (%)				70	60	50	20	40	60	30	10	50	80	40
Total of gravity			30	23	19	16	3	13	21	8	5	16	28	15

In order to extend the analysis of the reproducibility and quality of the simulation studies reviewed, a specific methodology has been implemented. Within this framework, two indices have been developed to provide a systematic and objective evaluation. These indices aim to analyze, on one hand, the frequency of identified gaps, and on the other hand, their severity:

Frequency: This index represents the occurrence rate of gaps across the studies listed in Table 3. It is calculated as the number of occurrences of each gap relative to the total number of gap, expressed as a percentage. The frequency ranges from 0% (none of the possible 10 gaps is present in the study) to 100% (all the possible 10 gaps are present in the study). This metric provides a quantitative measure of how widespread a specific gap is within the reviewed literature.

Severity: This index describes the relative impact of each gap on the reproducibility and quality of DES projects in MVCs. A severity score is assigned to each gap, ranging from 1 to 5, where:

1 corresponds to a minimally severe gap.

2 correspond to a moderately severe gap.

3 correspond to a severe gap.

4 correspond to a very severe gap.

5 correspond to an extremely severe gap.

By combining these indices, researchers can assess not only how commonly each gap occurs but also its relative importance in influencing the reliability and effectiveness of DES studies in MVCs. This dual evaluation helps prioritize which gaps require immediate attention to improve the field’s methodological rigor.

Among the studies, those by Asgary et al.^40,45,49 and Gupta et al.⁴¹ exhibit the highest frequency and severity of gaps, including critical omissions in simulation project phases such as conceptual model development, data parameterization, and results analysis. These gaps significantly affect the quality, reproducibility, and generalizability of the findings. In contrast, studies by Sala et al.,⁴⁷ Washington,⁴³ and Jerbi and Masmoudi⁴⁶ demonstrate fewer and less severe gaps, reflecting more comprehensive reporting and adherence to simulation best practices. The gaps in these studies are minor, involving occasional omissions or inconsistencies with limited impact on overall quality.

This disparity underscores the urgent need for standardized frameworks and consistent reporting practices in DES research for MVCs. High-frequency, high-severity gaps, as seen in studies by Asgary et al.,^40,45,49 highlight the importance of improving methodological rigor to ensure robust outcomes. Conversely, studies like those by Sala et al.,⁴⁷ Washington,⁴³ and Jerbi and Masmoudi⁴⁶ can serve as benchmarks for future research.

In summary, it is worth noting that the temporal focus of simulation studies on MVCs primarily aligns with the emergence of pandemics, such as the COVID-19 pandemic. This temporal association suggests a reactive rather than proactive approach to simulation research in the context of mass vaccination. While this may be understandable given the urgent nature of the pandemic response, it also highlights a missed opportunity for pre-emptive planning and preparedness. Moreover, the geographic distribution of these simulation studies predominantly in the Americas and Europe raises questions about the validity of their findings in other regions, particularly those with different healthcare systems, infrastructure, and socio-economic contexts. The limited representation of studies from Asia and Africa underscores the need for more inclusive and region-specific research to account for diverse healthcare settings and population demographics.

Furthermore, the predominance of real-life scenarios in simulation studies, as opposed to hypothetical scenarios, reflects a practical approach to modeling MVC operations. However, the reliance on real-life data introduces inherent limitations related to data availability, quality, and reliability. Future studies could benefit from exploring a balance between real-life and hypothetical scenarios to account for various contingencies and uncertainties. In addition, while simulation studies typically aim to replicate real-world conditions as accurately as possible, the assumption of constant service times in some studies overlooks the inherent variability and unpredictability of vaccination processes. Incorporating stochastic elements into simulation models can better capture the dynamic nature of MVC operations and improve the validity of simulation results. Another important factor to consider is the need for interdisciplinary collaboration in simulation research. Given the multifaceted nature of pandemics and the diverse range of stakeholders involved in mass vaccination operations, it is critical to bring together experts from fields such as epidemiology, public health, operations research, and computer science. Interdisciplinary collaboration can enrich simulation studies by integrating diverse perspectives and expertise, leading to more comprehensive and nuanced insights.

7. Conclusion

This study highlights that DES research on MVCs during pandemics is an emerging field, with studies focusing on various objectives such as optimizing MVC layouts, assessing staffing needs, and evaluating performance under fluctuating arrival rates. However, several critical gaps hinder the reproducibility, accuracy, and quality of findings in MVC DES studies. The most significant issues identified include insufficient reporting on key phases of the simulation projects, inconsistent data parameterization, and a lack of standardization across studies. The absence of solid conceptual models and variability in data sources further limits the ability to generalize results across different settings.

To become a reliable tool in pandemic response, these gaps in DES application in MCVs must be addressed. Recommendations include improving reporting standards, incorporating real-world variability in data parameterization, and fostering collaboration among researchers and public health practitioners to develop comprehensive, adaptable simulation models. Ensuring data quality and transparency will also be essential for enhancing the credibility and robustness of DES applications in MVCs. Addressing these issues will contribute to more effective MVC management and better preparation for future public health emergencies.

Supplemental Material

sj-docx-1-sim-10.1177_00375497241308563 – Supplemental material for Using discrete-event simulation for planning and managing mass vaccination centers: a comprehensive examination

Supplemental material, sj-docx-1-sim-10.1177_00375497241308563 for Using discrete-event simulation for planning and managing mass vaccination centers: a comprehensive examination by Abdessalem Jerbi in SIMULATION

Footnotes

Funding

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

ORCID iD

Abdessalem Jerbi

Supplemental material

Supplemental material for this article is available online.

Author biography

Abdessalem Jerbi, an associate professor at the Higher Institute of Industrial Management, University of Sfax, Tunisia, specializes in industrial and quality management with a research focus on optimizing manufacturing systems, supply chains, and medical systems. His expertise spans discrete-event simulation, flexible manufacturing systems, supply chain management, optimization methodologies, scheduling rules, and design of experiments. Dr. Jerbi’s work tackles complex challenges in these domains by enhancing efficiency, adaptability, and sustainability through advanced modeling techniques and simulation tools, driving innovative solutions in production scheduling, resource allocation, and supply chain coordination.

References

Huang

Wang

, et al. Clinical features of patients infected with 2019 novel coronavirus in Wuhan, China. Lancet 2020; 395: 497–506.

WHO, https://www.who.int/news-room/detail/27-04-2020-who-timeline---covid-19 (2020, accessed 16 August 2024).

WHO, https://www.who.int/director-general/speeches/detail/who-director-general-s-opening-remarks-at-the-media-briefing-on-covid-19---11-march-2020 (2020, accessed 16 August 2024).

Ganie

Wani

Yadav

MP.

Impact of COVID-19 outbreak on the stock market: an evidence from select economies. Bus Perspect Res. Epub ahead of print 8 February 2022. DOI: 10.1177/22785337211073635.

Zhang

Financial markets under the global pandemic of COVID-19. Finance Res Lett 2020; 36: 101528.

Polack

Thomas

Kitchin

, et al. Safety and efficacy of the BNT162b2 mRNA Covid-19 vaccine. N Engl J Med 2020; 383: 2603–2615.

Antonelli

Litwin

Stadnicka

Multiple system dynamics and discrete event simulation for manufacturing system performance evaluation. Proc CIRP 2018; 78: 178–183.

Helleno

Pimentel

Ferro

, et al. Integrating value stream mapping and discrete events simulation as decision making tools in operation management. Int J Adv Manuf Technol 2015; 80: 1059–1066.

Tako

Robinson

The application of discrete event simulation and system dynamics in the logistics and supply chain context. Decis Support Syst 2012; 52: 802–815.

10.

Melman

Parlikad

Cameron

Balancing scarce hospital resources during the COVID-19 pandemic using discrete-event simulation. Health Care Manag Sci 2021; 24: 356–374.

11.

Dieleman

Myles

Bulfone

, et al. Cost-effectiveness of routine transoesophageal echocardiography during cardiac surgery: a discrete-event simulation study. Br J Anaesth 2019; 124: 136–145.

12.

Ouda

Sleptchenko

Simsekler

MCE

. Comprehensive review and future research agenda on discrete-event simulation and agent-based simulation of emergency departments. Simul Model Pract Theory 2023; 129: 102823.

13.

Liu

Triantis

, et al. The diffusion of discrete event simulation approaches in health care management in the past four decades: a comprehensive review. MDM Policy Pract 2020; 5:2381468320915242.

14.

Caro

Briggs

Siebert

, et al. Modeling good research practices—overview: a report of the ISPOR-SMDM modeling good research practices Task Force–1. Med Decis Making 2012; 32: 667–677.

15.

Karnon

Stahl

Brennan

, et al. Modeling using discrete event simulation: a report of the ISPOR-SMDM modeling good research practices Task Force–4. Med Decis Making 2012; 32: 701–711.

16.

Kelton

Sadowski

Sturrock

DT.

Simulation with arena. 6th ed. New York: McGraw Hill, 2015.

17.

Rossetti

MD.

Simulation modeling and arena. 2nd ed. Hoboken, NJ: Wiley, 2021.

18.

Roberts

Russell

Paltiel

, et al. Conceptualizing a model: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-2. Value Health 2012; 15: 804–811.

19.

Robinson

Brooks

Kotiadis

, et al. Conceptual modeling for discrete-event simulation. Boca Raton, FL: CRC Press, 2010.

20.

Skoogh

Perera

Johansson

Input data management in simulation—industrial practices and future trends. Simul Model Pract Theory 2012; 29: 181–192.

21.

Thas

Graphical tools. In: Comparing distributions. Springer Series in Statistics. New York: Springer, 2010, pp. 201–209.

22.

Dodge

Chi-square goodness of fit test. In: The concise encyclopedia of statistics. New York: Springer, 2008, pp. 72–76.

23.

Nikitina

Chernukna

. Methods for nonparametric statistics in scientific research. Overview. Part 1. Theory Pract Meat Process 2021; 6: 151–162.

24.

Arena, https://www.rockwellautomation.com/en-us/products/software/arena-simulation.html (2024, accessed 16 August 2024).

25.

Simul8, https://www.simul8.com/ (2024, accessed 16 August 2024).

26.

AnyLogic, https://www.anylogic.fr/ (2024, accessed 16 August 2024).

27.

Simio, https://www.simio.com/ (2024, accessed 16 August 2024).

28.

SimPy, https://simpy.readthedocs.io/en/latest/ (2024, accessed 16 August 2024).

29.

Sargent

RG.

Verification and validation of simulation models. J Simul 2013; 7: 12–24.

30.

Jang

Choi

Verification method for discrete-event simulation based on discrete-event system formalism. Int J Ind Eng Theory Appl Pract 2023; 30.

31.

Eddy

Hollingworth

Caro

, et al. Model transparency and validation: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-7. Med Decis Making 2012; 32: 733–743.

32.

Briggs

Weinstein

Fenwick

, et al. Model parameter estimation and uncertainty analysis: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force Working Group-6. Med Decis Making 2012; 32: 722–732.

33.

Doudareva

Carter

Discrete event simulation for emergency department modeling: a systematic review of validation methods. Oper Res Health Care 2022; 33: 100340.

34.

Sargent

RG.

An interval statistical procedure for use in validation of simulation models. J Simul 2015; 9: 232–237.

35.

Kemp

Discrete-event simulation: modeling, programming, and analysis. J R Stat Soc Ser D 2003; 52: 408–409.

36.

Jerbi

Ammar

Krid

, et al. Performance optimization of a flexible manufacturing system using simulation: the Taguchi method versus OptQuest. SIMULATION 2019; 95: 1085–1096.

37.

Cumming

Understanding the new statistics: effect sizes, confidence intervals, and meta-analysis. New York: Routledge, Taylor and Francis Group, 2013.

38.

Mishra

Singh

Pandey

, et al. Application of student’s t-test, analysis of variance, and covariance. Ann Card Anaesth 2019; 22: 407–411.

39.

Moher

Shamseer

Clarke

, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) statement. Syst Rev 2015; 4: 1.

40.

Asgary

Najafabadi

Karsseboom

, et al. A drive-through simulation tool for mass vaccination during COVID-19 pandemic. Healthcare 2020; 8: 469.

41.

Gupta

Evans

Heragu

SS.

Simulation and optimization modeling for drive-through mass vaccination—a generalized approach. Simul Model Pract Theory 2013; 37: 99–106.

42.

Pilati

Tronconi

Nollo

, et al. Digital twin of COVID-19 mass vaccination centres. Sustainability 2021; 13: 7396.

43.

Washington

ML.

Evaluating the capability and cost of a mass influenza and pneumococcal vaccination clinic via computer simulation. Med Decis Making 2009; 29: 414–423.

44.

Beeler

Aleman

Carter

MW.

A simulation case study to improve staffing decisions at mass immunization clinics for pandemic influenza. J Oper Res Soc 2014; 65: 497–511.

45.

Asgary

Najafabadi

Wendel

, et al. Optimizing planning and design of COVID-19 drive-through mass vaccination clinics by simulation. Health Technol 2021; 11: 1359–1368.

46.

Jerbi

Masmoudi

Simulation modeling assessment and improvement of a COVID-19 mass vaccination centre operations. SIMULATION 2023; 99: 553–572.

47.

Sala

D’Urso

Giardini

Discrete-event simulation study of a COVID-19 mass vaccination centre. Int J Med Inf 2023; 170: 104940.

48.

Angelopoulou

Paul

. Discrete event simulation of a drive-through COVID-19 mass vaccination model: senior population prioritization strategies. In: 33rd European modeling & simulation symposium, Online 15–17 September 2021.

49.

Asgary

Blue

Cronemberger

, et al. Simulating a hockey hub COVID-19 mass vaccination facility. Healthcare 2022; 10: 843.

50.

Wang

Pan

Liu

, et al. Optimization of vaccination clinics to improve staffing decisions for COVID-19: a time-motion study. Vaccines 2022; 10: 2045.

51.

Lee

Chen

Pietz

, et al. Modeling and optimizing the public-health infrastructure for emergency response. Interfaces 2009; 39: 476–490.

52.

Pitman

Fisman

Zaric

, et al. Dynamic transmission modeling: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force Working Group-5. Med Decis Making 2012; 32: 712–721.

53.

Siebert

Alagoz

Bayoumi

, et al. State-transition modeling: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-3. Med Decis Making 2012; 32: 690–700.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.03 MB