Understanding Health Care Administrators’ Data and Information Needs for Decision Making during the COVID-19 Pandemic: A Qualitative Study at an Academic Health System

Abstract

Objective. The COVID-19 pandemic created an unprecedented strain on the health care system, and administrators had to make many critical decisions to respond appropriately. This study sought to understand how health care administrators used data and information for decision making during the first 6 mo of the COVID-19 pandemic. Materials and Methods. We conducted semistructured interviews with administrators across University of Florida (UF) Health. We performed an inductive thematic analysis of the transcripts. Results. Four themes emerged from the interviews: 1) common types of health systems or hospital operations data; 2) public health and other external data sources; 3) data interaction, integration, and exchange; and 4) novelty and evolution in data, information, or tools used over time. Participants illustrated the organizational, public health, and regional information they considered essential (e.g., hospital census, community positivity rate, etc.). Participants named specific challenges they faced due to data quality and timeliness. Participants elaborated on the necessity of data integration, validation, and coordination across different boundaries (e.g., different hospital systems in the same metro areas, public health agencies at the local, state, and federal level, etc.). Participants indicated that even within the first 6 mo of the COVID-19 pandemic, the data and tools used for making critical decisions changed. Discussion. While existing medical informatics infrastructure can facilitate decision making in pandemic response, data may not always be readily available in a usable format. Interoperable infrastructure and data standardization across multiple health systems would help provide more reliable and timely information for decision making. Conclusion. Our findings contribute to future discussions of improving data infrastructure and developing harmonized data standards needed to facilitate critical decisions at multiple health care system levels.

Highlights

The study revealed common health systems or hospital operations data and information used in decision making during the first 6 mo of the COVID-19 pandemic.

Participants described commonly used internal data sources, such as resource and financial reports and dashboards, and external data sources, such as federal, state, and local public health data.

Participants described challenges including poor timeliness and limited local relevance of external data as well as poor integration of data sources within and across organizational boundaries.

Results suggest the need for continued integration and standardization of health data to support health care administrative decision making during pandemics or other emergencies.

Keywords

administrative decision-making critical decision method COVID-19 pandemic data-driven decision-making

Introduction

Background

The coronavirus disease 2019 (COVID-19) was first confirmed on January 7, 2020.¹ The United States confirmed its first case within 13 days, and by mid-March, every state observed at least 1 known case.^1,2 Within 4 weeks, the incidence rate went from 0.01 per 100,000 to 43.55 per 100,000,³ and the hospitalization rates increased nearly 10-fold.⁴ The US health care system faced significant challenges in expanding its capacity to manage and treat COVID-19 patients while ensuring the adequacy of basic supplies.⁵ For example, testing supply shortages affected many areas of hospital administration and management. Limited testing capabilities and week-long turnarounds for test results exacerbated strains on managing other hospital resources such as bed capacity, personal protective equipment (PPE) supplies, and staffing.⁵ Increased demand and disruptions in the supply chain further exacerbated PPE shortages,⁶ with administrators reporting 1 to 6 mo of delays with 4- to 15-fold increases in costs.^5,7 Simultaneously, nonemergency procedures and appointments were canceled either by choice or circumstance, significantly reducing revenues for health care systems.^8,9 As cases and hospitalization rates continued to rise, the increasing logistic and financial strains coupled with the novelty of the COVID-19 virus placed enormous decision-making strain on health care administrators.

Significance

Many COVID-19 prediction models and dashboards were developed early in the pandemic to support health systems and public health planning.^10–13 A number of health systems shared knowledge and best practices and innovated to overcome challenges, such as surges in hospital admissions and distributing limited resources.^14–16 The scientific community generated much-needed information on COVID-19 as quickly as possible, as evident by the weekly average of 1486 new publications during the first 6 mo of the pandemic.¹⁷ As the pandemic raged on, data played a prominent role in facilitating decision making.^18–22 The growing body of COVID-19 literature,²³ in addition to the rapidly evolving state and federal guidance and supply chain uncertainties, resulted in health care administrators having to fit a large, ongoing stream of information into their assessments of a continually evolving situation. Health care administrators combined new information with their previous experiences and knowledge to make resource allocation, financial, and other critical decisions for their organizations. Despite the important implications of these administrative decision-making processes, few articles have explored how health care administrators used data to understand the evolving pandemic and make operational decisions. When examining data utilization for COVID-19 decision making, prior research has been on clinical decision making in treating patients, either in general^24–28 or for specific conditions and populations.^29–32 Literature has also focused on strategies, concepts, and principles to assist with clinical decision making^33–39 and reflection for future clinical directions in treating COVID-19.^40–43

Given that knowledge gap, the objective of this study is to investigate how health care administrators used data and information to navigate the novel operational health care situation in the acute phase (first 6 mo) of the COVID-19 pandemic. The current body of literature provides insight into how hospitals and health systems responded and innovated,^14–16 how public health officers make decisions for shelter-in-place mandates,⁴⁴ and how government leaders and leaders in other industries dealt with the pandemic.⁴⁵ However, a gap in the literature exists in understanding the patterns in cognition and information in how individual health care administrators made decisions during that challenging time. We addressed this objective by conducting a qualitative study using the critical decision method (CDM) approach to elicit knowledge and experiences from health care administrators early in the COVID-19 pandemic.

Materials and Methods

Recruitment

Semistructured interviews using the CDM approach were conducted to understand patterns in cognition, information, and decision making by evaluating gaps in operational data and information technology (IT) systems. Participants were recruited across University of Florida (UF) Health. UF Health is an academic health system with 4 primary hospitals and 2 faculty group practices spread across 3 campuses in Central and North Florida. Two of the hospitals are teaching focused and have level 1 trauma centers, with one also being a safety net hospital serving predominantly underinsured and uninsured patients. The other 2 hospitals are community hospitals with affiliated outpatient and ambulatory services. Given this mix of hospitals and physician practices in different locations (Gainesville, Jacksonville, Leesburg/The Villages) spanning different settings (rural, suburban, and urban), UF Health utilizes a matrix organizational structure. Each hospital and faculty practice has its executive leadership team that reports to a health system president. To obtain diverse perspectives across the health system, we purposefully recruited to ensure leaders from each primary hospital and faculty practice participated and to ensure each campus was represented by at least 4 participants. We also purposefully recruited for diversity in participants’ current administrative job roles.

We emphasized recruiting administrative and clinical leaders (e.g., executive suite, vice presidents, chief clinicians, medical/disaster directors) who made critical policy and process decisions for their organization during the COVID-19 pandemic. Potential participants were invited to participate in the interview via email from study team members. Before scheduling the interview, potential participants received an information sheet about the study and an informed consent document. Participants were provided time to review, ask questions, and determine if they consent to participate. Administrative personnel then followed up to set up the interview. The study was approved by the UF Institutional Review Board (IRB202001126).

Semistructured interviews

Semistructured interviews were conducted to elicit how participants made decisions during the acute phase of the COVID-19 pandemic. Following the CDM approach, participants were requested to recall the timeline of the pandemic response and key operational policies or process changes they were involved in (see Appendix I for the interview guide). They were then asked probing questions about critical decisions made for the COVID-19 response, such as personal protective equipment (PPE) management, telehealth use, COVID-19 screening and testing, and laboratory supply management. Questions also explored the types of data that administrators used to make such decisions and information that would have been useful when making decisions but was not available at the time.

Interviews lasting approximately 45 to 60 min were conducted between July and September 2020. The principal investigator, a PhD-trained professor experienced in qualitative research (C.A.H.), conducted the individual interviews. A study team member (C.G. or C.M.) was present to take field notes at each interview. Three interviews were conducted in person, with the remaining conducted via Zoom to adhere to safety guidelines. All interviews were audiotaped and recorded. Datagain (Vienna, VA) transcribed the interview recordings. To preserve confidentiality, 2 study team members (C.M. and T.M.) deidentified the transcribed interviews by removing names, specific titles, and site locations before analysis.

Analysis

We conducted an inductive thematic analysis, as we focused on interpreting the data and then building consensus.⁴⁶ Initially, 3 analysts (C.G., C.M., and T.M.) reviewed 4 transcripts from 2 separate sites to identify topics of interest and potential codes related to the research aims. We read each transcript line by line, cut out meaningful quotes, extracted them, and began sorting and combining similar quotes and topics of interest related to our study objective (first-level coding). The principal investigator (C.A.H.) met with the analysts to discuss similarities and differences in preliminary codes, refine definitions, and merge codes as appropriate to develop a preliminary codebook. The analysts independently re-reviewed and recoded the original 4 transcripts and 2 additional transcripts from the third location based on the preliminary codebook. The analysts reviewed each other’s coding and noted any discrepancies. Discrepancies were discussed and addressed over meetings. If consensus was not reached, the analysts met with the principal investigator and resolved differences as a group. Some codes were revised, and the analysts recoded the transcripts accordingly. For the remaining 9 transcripts, each analyst coded 3 transcripts and reviewed 3 other transcripts that another analyst coded.

Results

Participants

Fifteen of 16 administrative and clinical leaders responded to an invitation and participated in the study. The participants came from 3 campuses: UF Health Shands (n = 5), UF Health Jacksonville and UF Health North (n = 6), and UF Health Central Florida (n = 4). The participants were predominantly male (n = 12). The study participants represented different educational backgrounds: health care and/or business administration (n = 5), medicine (n = 8), such as emergency medicine, infectious disease, family medicine, and pathology, pharmacy (n = 1), and health informatics (n = 1). Their experience in the health care industry averaged 30 y, and their roles spanned across different organizational levels: executive suite officers (n = 10), presidents/vice presidents (n = 3), and directors (n = 2). All participants had responsibilities of key areas of COVID-19 response, including telehealth, laboratory services, and supply chain management, in addition to general health care management responsibilities. After coding the interviews from this diverse group, we judged that no new codes were emerging and thus stopped recruitment.

Main Themes

The final codebook included 37 codes (22 parent codes and 15 child codes). The principal investigator and the analysts chose to focus on the 8 codes most related to the research aims for the thematic analysis. The team reviewed each extracted passage in 1 code to identify emerging themes (secondary level coding) and reached a consensus on 3 initial themes. Two analysts (C.G. and C.M.) reviewed passages from the remaining 7 codes to categorize passages into the 3 initial themes and additional emerging themes as appropriate.

Overall, 4 themes emerged from our qualitative thematic analysis, which provided insights into health care administrators’ data and information needs for critical decision making during the COVD-19 pandemic. Those 4 themes are 1) common types of individual health systems or hospital operations data; 2) public health and other external data sources; 3) data interaction, integration, and exchange; and 4) novelty and evolution in data, information, or tools used over time. Those themes are described in more detail and illustrated with participant quotes here.

Common types of individual health systems or hospital operations data

All participants extensively described internal operations and financial data as essential to support decision making. Common data cited by participants as vitally important included hospital census, bed availability, outpatient clinic volumes, number of scheduled surgeries, ventilator availability, number of positive COVID-19 patients, number of suspected COVID-19 cases (PUI), number of patients holding (awaiting inpatient bed availability) in the emergency department (ED), PPE inventory, testing supply inventory, medication supply, staffing availability, and financial cash flow.

We get a report every day on how many patients are in-house you know the PUIs and how many were positive, how many were negative and how many folks are in you know either ICU or non-vents. So, that goes out every morning as well to all the physicians so, they know what to do. (Participant 13)

You’ll see that we have data on. . . . With our PPE or testing supplies, you’ll see we have very detailed data on what we currently have, what the projected usage is and therefore how many days of each of these different aspects. (Participant 9)

In the interviews, participants from 3 locations mentioned how hospital incident command systems (HICS) were essential to centralizing communication, quality control, and coordination of the information flow and operations during the COVID-19 pandemic. Many of the participants identified HICS as one of the first steps they took in the early phase of the pandemic.

Renamed it the COVID Operations Hub and the point it still functioned completely like a Command Center, but it was a central place where all information, the intent was all information would flow into that Hub and then we would disseminate information out in a very organized, centralized and consistent way. But it also served as a resource for all of our staff. (Participant 15)

Dashboards were another widespread tool used by administrators to track and manage resources within the health system. The dashboards displayed aggregated information from multiple areas of the organization, historical data, and predictions (e.g., case volume, bed capacity, and PPE use).

I think the dashboard has been largely helpful for us, you know, actually is a single source of information as like, no matter when I look at it. So I’m looking at, you know, both the inpatients and the ICU trend, PPE trend, looking at what is going on in the region. (Participant 14)

Second, certainly during the pandemic, we’ve looked at the dashboard, which helps us look at our COVID volume, and how that looks in terms of again, what’s on the [masked location] Campus, [masked location] Campus, ICU capacity. So, there’s different things on that dashboard. And we look at our predictive curves to try and get a sense of “Are we matching them? Where are we headed? Where’s our ventilator capacity, the ICU capacity, PPE, which we’ve now taken off?” We do look at our testing kits. So, that helps us make decisions around COVID, which is our current activity. (Participant 6)

Public health and other external data source

In response to the COVID-19 pandemic, national, state, and local public health agencies began collecting, tracking, and disseminating information about COVID-19 cases and guidelines for hospitals and health systems. Administrators indicated that local/regional information was also useful and compared and contrasted it with their individual health system data.

And we were really at that time, being guided by what was coming out of the CDC [Centers for Disease Control and Prevention]. I think the Florida department of health was very early on, although we had good communication with the [masked] County department of health. So, we were speaking to them on a daily basis. So, the CDC, the [masked] County department of health, and we were really early on looking at the early the, uh, IHME [Institute for Health Metrics and Evaluation] predictions. (Participant 2)

As COVID-19 spread across the country, administrators conferred with other health care systems to determine what others might be doing and coordinate COVID-19 responses locally.

I do think when we were trying to figure out what policies to put into place or protocols or algorithms, we did reach out to colleagues a fair amount or what we would say are similar academic institutions and we would try to understand what were they doing, what great idea did they have. (Participant 15)

One of the other things that we are using now, or that I’m finding helpful now, is the hospitalization rate at all the other hospitals in the city. . . . One of the data points that I’m using now is the hospitalization rates for COVID in these other hospitals to give me an idea of whether or not we are truly seeing a decline in hospitalizations, or we’re still at that peak. (Participant 5)

While administrators used external information to facilitate data-driven operational decisions, participants suggested various aspects of data quality, including timeliness or relevance (e.g., to their campus or facility), were critical factors to consider. For example, participant 3 described how information or guidance from UF Health’s main campus leadership did not always apply locally.

So, we used CDC guidelines at the time, and then we also used other, other advice from [main campus leader]. Uh, [masked healthcare system]. One of the big things that came up was the, uh, decision and how to reprocess them. (Participant 3)

Yes, so knowing what other hospitals are doing. So, we get that today too. But again, it shows up at a different point in time. So, the city has an [masked name] dashboard that they require every hospital to put their information in. And usually, around 10:00 or 11:00 in the morning, I’ll get it. But that’s maybe 2 or 3 hours later when I need it, so to speak. Just so I can see. So, it’d be nice to see that. (Participant 6)

Data interaction, integration, and exchange

Data and information from both internal and external sources drove administrators’ decision making during the COVID-19 pandemic. However, processing information does not occur in isolation. Participants discussed the need for data integration and validation across different departments, organizations, agencies, and expertise.

I reviewed the actual literature. . . . The medical literature. And then for policies and procedures, I’ll, I’ll go to the CDC and the department of health. But then I’ll go to some of the top systems. I’ll see what [masked organization] is doing. [masked organization], [masked organization] early on. Some of them, [masked organization], and some of the people that, uh, in [masked location]. And then if I still have a question, I’ll, I’ll bounce it off the experts at [masked organization] . . . adapting those policies. SMEs, the subject matter experts first. And then after that, I’m gonna put it in the opposite—put—give it to the operators to see, can you implement this? Does this make sense? How hard is it gonna be to monitor? Right? So you could have the best policy, but if you can’t implement it and you can monitor, it doesn’t mean anything as a policy. (Participant 2)

A few participants also discussed the need to coordinate information across organizations to prepare for a regional response.

Data coordination among organizations. So, almost every day I text the [administrator] of [masked location] or he texts me our numbers. Right? And I suspect they do that in [masked location] and I suspect they do that in [masked location] and—and so, there is no good useful repository of data that makes regional sense. Like it’s nice to know what’s going on in [masked location] but I don’t care. Right? I need to care about the region I serve. And so, the [masked organization] has a nice website but that has zero utility to me. (Participant 11)

Data integration and coordination challenges are further compounded by reporting guidelines, privacy mandates, and the lack of interoperable systems.

So if you’re in [masked geographic location], and you have symptoms and you put down [masked location] resident, you get reported here. Likewise, if you’re a [masked occupation] and you go to CVS (pharmacy) and get tested, but you put a [masked location] county address down as your home, your report gets sent there. So we have to cross-match the state list with our list and that creates a lot of challenges. . . . And then also the privacy issues between the Department of Health and everybody else. They have so many firewalls. So you can’t break through to be able to easily access the data. So access to the existing data in the public domain is exceptionally difficult. (Participant 10)

So, there’s almost no interoperability between health systems. We don’t even have an in between our own health system right, we have the rehab hospital and the LTAC (long-term acute care) units owned by [masked organization] or in partner with [masked organization]. They have their own version Epic [electronic health record system] and you have to sign out at two different health systems. You can’t actually communicate between the two. So, whatever people thought and for me Healthcare Information Exchange was going to be in most of our opinions, it’s been a total failure. (Participate 7)

Intraoperability is particularly important to administrators. Participants expressed the need to integrate data within the organization that typically exists in different functional departments and systems.

Well, I think COVID has actually stimulated us to create some new, more real-time data streams and have us break some of that data up in different ways. And so, we get out of both—out of the databases that feed off of our real-time systems, both billing systems and clinical systems and financial systems that then get assembled into a series of dashboards that feed me sometimes financial information on a dashboard about our enterprise level and at departmental levels and at subdepartmental levels. Production data on a dashboard. Production data would be RVU [relative value unit] data or visit data or procedure data, or things like that, broken out in the same way. . . the data support people that I have, create a lot of ad hoc views for me when I’ve got a specific question about, what’s going on with this service, or where does this set of patients come from or where does this set of patients go? And then they will create that for me, pretty much on the fly. (Participant 9)

So, we are getting pieces of information during the course of the day. But right now, it’s also not—And future development, I think there’s going to be some app that we can get to. Because right now, you’re getting it from multiple sources, right? So, [masked department] have sent me some stuff, [masked department] some stuff. [masked department] is sending the links. Everybody is sending lots of different pieces of information. And it’d be nice to go to one particular source, one app that we could use. Let me just dial in and see what’s going on today. (Participant 6)

Administrators often have to consider information from various functions, such as finances, supply chain, and testing, to respond to the COVID-19 pandemic appropriately.

I guess I would ask, I don’t know if you can think just about labs—it’s hard, because it’s all intermingled, but how do you weigh the tradeoffs of having supplies today versus having supplies tomorrow? Cash; having the financial implications of getting a bunch of supplies, if we can have them? How do you or your teams work through those tradeoffs? (Participant 5)

And so she would pull a group of all the important stakeholders together to make sure that we weren’t leaving out nursing, or certain physician groups, or the ED, or laboratory, IT, to make sure that everybody was there when decisions were made. (Participant 6)

Novelty and evolution in data or tools used over time

Data and information streams needed to evolve as the immense amount of COVID-19 information was processed. Specifically, participants voiced a desire for some filtering of data to that most salient for their individual organization.

It [information] went from a drip to the faucet being on to a fire hose in about 3 or 4 weeks. (Participant 4)

Initially, we just didn’t have enough information. And then there came a time—I don’t know how many weeks into it, where we had a whole bunch of information, lots of information, and it got put onto the dashboard. After a month or two, some of us began to recognize we weren’t looking at about two-thirds of the dashboard because it just didn’t impact what we were thinking or decision making. After that, we didn’t really need any more information because we had a sense of what the depth of our ICU capacity was, so a lot of this got abbreviated. I’m just talking about your mental filter, what you took away, and needed to deal with. (Participant 4)

During the first 6 mo of the COVID-19 pandemic, data-driven decisions evolved based on the operational needs or resource constraints at a specific moment in time.

Then, the shortage of PPE; we added in PPE numbers and CPE [collective protective equipment] supply. And then, when we were really short on swabs, we added in days’ supply of swabs on hand so that our providers can understand that this is where we are and what we need to focus on to work on trying to get more swabs. Then all of a sudden, we’ve got 10,000 swabs. So, I think that’s when I said, take that off the indicator, we don’t need that on the dashboard anymore. But what we do need is day supply of reagents. So, how many days can we go running these tests now that we have all the swabs we want? So, it has evolved over time, and every one of those indicators at that moment in time was useful to us in our HICS [hospital incident command system] meetings, in our planning, and in our strategy. (Participant 5)

So that was a way where those decisions were made well. And to be honest with you, I felt very good about how those decisions were made. They were unfortunate and they were painful, but I thought that the process of making those decisions was driven by data in terms of availability, with testing capabilities, options for testing and what we could do to flex and change. And that changed practice. So we were able to 3D print swabs. We were able to, I think at one point, 3D-printed test tubes that we were short on. Then also were able to be one of the pioneers in the country in terms of pool testing, which is a different technique of, you may have heard of it, but where you get numerous specimens likely to be negative, put them and test them as one, and if it’s positive, you go back and test each one individually. And you do that at scale, you can really increase your testing capability and not tap your limited resources quite a bit. So it makes it a lot easier to do that way. (Participant 10)

Throughout the acute phase of the COVID-19 pandemic, administrators indicated a shift in how they made decisions.

We used to have a daily call with [masked location] with the command center there. And then now what’s once a week and then if I have specific issues or questions, I’ll run it up to either [executive team member] or [executive team member] to see what’s going on. And also I’ve connected to the [masked department] people there, the critical care people. (Participant 2)

It’s more of there were a lot of people making decisions before that kind of worked at the time. But now it’s the kind of core group of people making some day-to-day kind of decisions has to be smaller in order to move forward. (Participant 8)

Discussion

This study used semistructured interviews with the CDM approach to help understand how health care administrators made decisions early in the COVID-19 pandemic. Specifically, our interviews with 15 health care administrators provided insights on how they used data and information to navigate the enormous decision-making strains during the first 6 mo of the COVID-19 pandemic. The study revealed common health systems or hospital operations data that health care administrators used to guide critical decisions during the COVID-19 pandemic. While they relied on internal data, information and data from public health and other external organizations were also considered. Given the need to draw upon data and information from internal and external sources, health care administrators discussed how data interact across boundaries and the necessity of integration and a more liberal exchange of data from different sources. However, they also advised that the quality, reliability, and generalizability of external data be considered. With the novelty of COVID-19 and the evolution of external market forces, health care administrators commented on the changes in data or tools they have used over time in their decision-making process.

To effectively respond to a pandemic, health care administrators need clinical information (e.g., case rates and lab results) and organizational resource information (e.g., available beds, staff, and supplies). Some of these needs may be met by information within administrators’ organization, while others require extraorganizational information. A shared electronic medical record system⁴⁷ or an interoperable framework for data sharing across organizations^48,49 can help provide such integrated data. Nevertheless, such a robust data-sharing infrastructure does not currently exist to support administrators in United States health care organizations.⁵⁰ Indeed, in the early months of the COVID-19 pandemic, our study participants reported relying on texting and phone calls to colleagues to help meet some of their key information needs. Thus, increased data standardization and integration may facilitate more efficient information sharing^49,51 and thus more timely and reliable decision making.⁴⁷

In addition, as integrated data grow in volume and velocity, they must be presented in a consistent and usable format so they are interpretable and actionable for important decisions.⁵² Our study found that several administrators relied heavily on dashboards to consume needed information. Moreover, the most important information for display on those dashboards changed over time as the pandemic evolved. Thus, supporting administrative decision making may benefit from ongoing evaluation of dashboards or other data-delivery mechanisms to ensure they meet current information needs.

The COVID-19 pandemic has created an unprecedented strain for the health care system, based on many factors, from pent-up demand to reduced cash flow.^5–9 Those factors resulted in a time of uncertainty and complexity in which health care administrators needed to make critical decisions, potentially with limited information, to effectively lead their organizations through the pandemic. Human creativity and innovation are essential to meet the organization’s needs and those of the community it serves while remaining sustainable in the long run. Given the shift in resource constraints and external forces over time, health care administrators must be nimble and evolve throughout the continuing COVID-19 pandemic.

Limitations

While this qualitative study provides insight into how health care administrations make decisions in the face of unprecedented challenges caused by the COVID-19 pandemic, our approach also has limitations. The participants were recruited from a single organization, and the sample size was modest, which means our results are not likely to transfer to all health care administrators. We mitigated this limitation by purposefully recruiting administrators across multiple campuses and facilities with unique characteristics. Each campus is located in a different setting (teaching hospitals and level 1 trauma centers v. community hospitals, rural v. metropolitan areas) and serves distinct patient populations. Notably, 1 of the 3 campuses and its 2 hospitals was newly acquired by the academic health system at the time of the study and retained much of their culture and operations. Thus, our findings likely represent many health care administrators’ experiences. Still, our findings may not transfer to other organizations, especially to smaller or non–academic-affiliated health systems. Conversely, qualitative interview research is not intended to produce representative, precise, or generalizable outcomes.⁵³ We believe our study elicited rich, detailed knowledge from diverse health care administrators regarding how data and information played a role in their critical decision making during the COVID-19 pandemic. Nevertheless, future studies involving for-profit, nonacademic organizations and health systems of different sizes and in different geographic areas may yield a broader set of important themes.

Conclusion

This study explored how health care administrators used data and information to make critical decisions during the COVID-19 pandemic. Fifteen administrative and clinical leaders from 3 separate locations within 1 academic health system discussed the common types of operations data and how public health data informed their decision making. Participants highlighted the challenges to integrating data and information across intraorganizational and interorganizational boundaries. Even within the first 6 mo of the pandemic, health care administrators revealed changes in data, information, and tools to make critical decisions. These key themes will contribute to future discussions of improving data infrastructure and developing harmonized data standards needed to facilitate critical decisions at an organizational level, if not locally.

Supplemental Material

sj-doc-1-mpp-10.1177_23814683221089844 – Supplemental material for Understanding Health Care Administrators’ Data and Information Needs for Decision Making during the COVID-19 Pandemic: A Qualitative Study at an Academic Health System

Supplemental material, sj-doc-1-mpp-10.1177_23814683221089844 for Understanding Health Care Administrators’ Data and Information Needs for Decision Making during the COVID-19 Pandemic: A Qualitative Study at an Academic Health System by Christina Guerrier, Cara McDonnell, Tanja Magoc, Jennifer N. Fishe and Christopher A. Harle in MDM Policy & Practice

Footnotes

Acknowledgements

We acknowledge Jiang Bian, PhD, Francois Modave, PhD, Zhaoyi Chen, and Xing He from the Health Outcomes & Biomedical Informatics, College of Medicine, University of Florida, Gainesville, Florida; Gloria Lipori from the University of Florida Health Shands, Gainesville, Florida; and Daniel Norez and Ian Tfirn from the Center for Data Solutions, University of Florida Health Science Center, Jacksonville, Florida.

Authors’ Note

The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article. The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Financial support for this study was provided entirely by the University of Florida Clinical and Translational Science Institute, which is supported in part by the NIH National Center for Advancing Translational Sciences under award No. UL1TR001427. The funding agreement ensured the authors’ independence in designing the study, interpreting the data, writing, and publishing the report. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

ORCID iDs

Christina Guerrier

Christopher A Harle

Supplemental Material

Supplementary material for this article is available on the MDM Policy & Practice website at .

References

Holshue

DeBolt

Lindquist

, et al. First case of 2019 novel coronavirus in the United States. N Engl J Med. 2020;382(10):929–36.

Geographic differences in COVID-19 cases, deaths, and incidence—United States, February 12–April 7, 2020. MMWR Morb Mortal Wkly Rep. 2020;69:465–471. http://dx.doi.org/10.15585/mmwr.mm6915e4.

Trends in number of COVID-19 cases and deaths in the US reported to CDC, by state/territory, Centers for Disease Control and Prevention. Available from: https://covid.cdc.gov/covid-data-tracker/?CDC_AA_refVal=https%3A%2F%2Fwww.cdc.gov%2Fcoronavirus%2F2019-ncov%2Fcases-updates%2Fcases-in-us.html#trends_totalandratecasessevendayrate. Accessed March 12, 2021.

COVID-NET. COVID-19-associated hospitalization surveillance network, Centers for Disease Control and Prevention. Available from: https://gis.cdc.gov/grasp/COVIDNet/COVID19_3.html. Accessed March 12, 2021.

Grimm

. Hospital experiences responding to the COVID-19 pandemic: results of a national pulse survey March 23–27, 2020. April 2020. Available from: https://oig.hhs.gov/oei/reports/oei-06-20-00300.pdf. Accessed March 12, 2021.

Bradsher

Alderman

. The world needs masks. China makes them, but has been hoarding them. New York Times. March 13, 2020. Available from: https://www.nytimes.com/2020/03/13/business/masks-china-coronavirus.html. Accessed March 12, 2021.

Deshpande

Staats

Swaminathan

, et al. Where did all the PPE go? The COVID-19 disconnect between hospitals and the healthcare supply chain. Kenan Insights. April 15, 2020. Available from: https://kenaninstitute.unc.edu/kenan-insight/where-did-all-the-ppe-go-the-covid-19-disconnect-between-hospitals-and-the-healthcare-supply-chain/. Accessed March 12, 2021.

Hospitals and health systems face unprecedented financial pressures due to COVID-19. American Hospital Association. May 2020. Available from: https://www.aha.org/guidesreports/2020-05-05-hospitals-and-health-systems-face-unprecedented-financial-pressures-due. Accessed March 12, 2021.

Fromberg

Krupa

. Kaufman Hall perspective: national hospital flash report. Kaufmann Hall. April 2020. Available from: https://flashreports.kaufmanhall.com/kha-perspective-april-2020. Accessed March 12, 2021.

10.

Johns Hopkins Coronavirus Resource Center. COVID-19 dashboard by the Center for Systems Science and Engineering (CSSE) at Johns Hopkins University. 2020. Available from: https://coronavirus.jhu.edu/map.html. Accessed March 12, 2021.

11.

Institute for Health Metrics and Evaluation. COVID-19 projections. 2020. Available from: https://covid19.healthdata.org/. Accessed March 12, 2021.

12.

Weissman

Crane-Droesch

Chivers

, et al. Locally informed simulation to predict hospital capacity needs during the COVID-19 pandemic. Ann Intern Med. 2020;173(8):680–81.

13.

American Hospital Association. A compendium of models that predict the spread of COVID-19. 2020. Available from: https://www.aha.org/guidesreports/2020-04-09-compendium-models-predict-spread-covid-19. Accessed March 12, 2021.

14.

Gupta

Federman

. Hospital preparedness for COVID-19 pandemic: experience from department of medicine at Veterans Affairs Connecticut Healthcare System. Postgrad Med. 2020;132(6):489–94.

15.

Shin

Abdelhalim

Chau

, et al. Responding to coronavirus disease 2019: LA County hospital experience. Emerg Radiol. 2020;27(6):785–90.

16.

Richmond

. The coronavirus pandemic: experience from a rural West Virginia tertiary care hospital. Am Surg. 2020;86(6):611–4

17.

Chen

Allot

. Keep up with the latest coronavirus research. Nature., 2020;579(7798):193.

18.

Paiva

Afonso

RJM

de Oliveira

, et al. A data-driven model to describe and forecast the dynamics of COVID-19 transmission. PLoS One. 2020;15(7):e0236386.

19.

Nadler

Wang

Arcucci

, et al. An epidemiological modelling approach for COVID-19 via data assimilation. Eur J Epidemiol. 2020;35(8):749–61.

20.

Villanustre

Chala

Dev

, et al. Modeling and tracking Covid-19 cases using Big Data analytics on HPCC system platformm. J Big Data. 2021;8(1):33.

21.

Hasan

Putri

ERM

Susanto

, et al. Data-driven modeling and forecasting of COVID-19 outbreak for public policy making. ISA Trans. 2021:S0019-0578(21)00030-6.

22.

Mayo Clinic COVID-19 Predictive Analytics Task Force, Pollock

Carter

Dowdy

, et al. Deployment of an interdisciplinary predictive analytics task force to inform hospital operational decision-making during the COVID-19 pandemic. Mayo Clin Proc. 2021;96(3):690–8.

23.

Chen

Allot

. LitCovid: an open database of COVID-19 literature. Nucleic Acids Res. 2021;49(D1):D1534–40.

24.

Leeuwenberg

Schuit

. Prediction models for COVID-19 clinical decision making. Lancet Digit Health. 2020;2(10):e496–7.

25.

Subudhi

Verma

Patel

. Prognostic machine learning models for COVID-19 to facilitate decision making. Int J Clin Pract. 2020;74(12):e13685.

26.

Metlay

Armstrong

. Clinical decision making during the COVID-19 pandemic. Ann Intern Med. 2021:M20–8179.

27.

Teixeira

Rosa

Rodrigues Filho

, et al. The medical decision-making process in the time of the coronavirus pandemic. Rev Bras Ter Intensiva. 2020;32(2):308–11.

28.

Debnath

Barnaby

Coppa

, et al. Machine learning to assist clinical decision-making during the COVID-19 pandemic. Bioelectron Med. 2020;6:14.

29.

Morray

Gordon

Crystal

, et al. Resource allocation and decision making for pediatric and congenital cardiac catheterization during the novel coronavirus SARS-CoV-2 (COVID-19) pandemic: a U.S. multi-institutional perspective. J Invasive Cardiol. 2020;32(5):E103–9.

30.

Borasio

Gamondi

Obrist

, et al. COVID-19: decision making and palliative care. Swiss Med Wkly. 2020;150:w20233.

31.

DeFazio

Kahan

Fallon

, et al. Development of pediatric surgical decision-making guidelines for COVID-19 in a New York City children’s hospital. J Pediatr Surg. 2020;55(8):1427–30.

32.

Bruce-Hickman

Fan

Plaat

, et al. Decision-making on the labour ward during the COVID-19 pandemic. Int J Obstet Anesth. 2021;45:150–1.

33.

Stavroglou

Ayyub

Kallinterakis

, et al. A novel causal risk-based decision-making methodology: the case of coronavirus. Risk Anal. 2021;41(5):814–30.

34.

Avila

Kahmann

Alho

, et al. Hemogram data as a tool for decision-making in COVID-19 management: applications to resource scarcity scenarios. PeerJ. 2020;8:e9482.

35.

Sevy Majers

Warshawsky

. Evidence-based decision-making for nurse leaders. Nurse Lead. 2020;18(5):471–5.

36.

Liao

Wen

. A consensus model to manage the non-cooperative behaviors of individuals in uncertain group decision making problems during the COVID-19 outbreak. Appl Soft Comput. 2021;99:106879.

37.

Canals Lambarri

. Learning from the COVID-19 pandemic: concepts for good decision-making. Rev Med Chil. 2020;148(3):418–20.

38.

Ferrinho

Sidat

Leiras

, et al. Principalism in public health decision making in the context of the COVID-19 pandemic. Int J Health Plann Manage. 2020;35(5):997–1000.

39.

Angeli

Montefusco

. Sensemaking and learning during the COVID-19 pandemic: a complex adaptive systems perspective on policy decision-making. World Dev. 2020;136:105106.

40.

Habli

Alexander

Hawkins

, et al. Enhancing COVID-19 decision making by creating an assurance case for epidemiological models. BMJ Health Care Inform. 2020;27(3):e100165.

41.

Burman

Cairns

Canestrini

, et al. Making ordinary decisions in extraordinary times. BMJ. 2020;370:m3268.

42.

Fischhoff

. Making decisions in a COVID-19 world. JAMA. 2020;324(2):139–40.

43.

Hilton

Flemyng

Soares-Weiser

. COVID-19: working together and making a difference for decision-makers. Cochrane Database Syst Rev. 2020;12:ED000150.

44.

Aragón

Cody

Farnitano

, et al. Crisis decision-making at the speed of COVID-19: field report on issuing the first regional shelter-in-place orders in the United States. J Public Health Manag Pract. 2021;27S1:S19–28.

45.

Visentin

Reis

Cappiello

, et al. Sensing the virus. how social capital enhances hoteliers’ ability to cope with COVID-19. Int J Hosp Manag. 2021;94:102820.

46.

Barbour

. Checklists for improving rigour in qualitative research: a case of the tail wagging the dog? BMJ. 2001;322(7294):1115–7.

47.

Salway

Silvestri

Wei

, et al. Using information technology to improve COVID-19 care at New York City Health + Hospitals. Health Aff (Millwood). 2020;39(9):1601–4.

48.

Gansel

Mary

van Belkum

. Semantic data interoperability, digital medicine, and e-health in infectious disease management: a review. Eur J Clin Microbiol Infect Dis. 2019;38(6):1023–34.

49.

Shanbehzadeh

Kazemi-Arpanahi

Mazhab-Jafari

, et al. Coronavirus disease 2019 (COVID-19) surveillance system: Development of COVID-19 minimum data set and interoperable reporting framework. J Educ Health Promot. 2020;9:203.

50.

Fidler

Gostin

. The WHO pandemic influenza preparedness framework: a milestone in global governance for health. JAMA. 2011;306(2):200–1.

51.

Leslie

, et al. Development of an openEHR template for COVID-19 based on clinical guidelines. J Med Internet Res. 2020;22(6):e20239.

52.

Gao

Tao

Huang

, et al. Management and data sharing of COVID-19 pandemic information. Biopreserv Biobank. 2020;18(6):570–80.

53.

Sandelowski

. Justifying qualitative research. Res Nurs Health. 2008;31(3):193–5.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.08 MB