Municipal performance management during the covid pandemic

Context

The COVID-19 pandemic has affected all aspects of human life. For many public organizations, the pandemic has transformed the assumptions and structures they previously operated under—from bureaucratic norms and contingency planning to emergency preparedness and political oversight (Ansell, Sørensen, and Torfing 2020; Carungu, Di Pietra, and Molinari 2021). The pandemic has also exposed institutional weaknesses and capacity gaps in dealing with such a crisis, suggesting the future challenges organizations are likely to face if they don’t rethink their traditional hierarchy (Dunlop, Ongaro, and Baker 2020).

A county child protective services official, for example, struggled to consider new ways to collect data on child abuse now that children were spending more time at home and having limited interactions with teachers, nurses, and other essential reporters of abuse. Similarly, some municipal waste management divisions had to consider altering their contracts with collection companies due to a sudden increase in residential waste as people were at home more (Hantoko et al., 2021). Parks were forced to issue physical-distancing protocols, and local libraries had to find virtual methods to continue offering services (Miller-Rushing et al., 2021). Social service agencies were besieged with welfare requests and struggled to provide food and shelter to everyone who needed it (Cohen, Russo, Campos, and Allegri 2020). Public employees frequently had to operate outside their prescribed roles and responsibilities to meet such unexpected challenges (Schuster et al., 2020).

The urgency of these issues has led to emerging research on the management implications of the COVID-19 pandemic in the public sector (e.g., Hantoko et al., 2021; Miller-Rushing et al., 2021; Sancino, Garavaglia, Sicilia, and Braga 2020). This literature suggests that the pandemic exacerbated insidious problems that already existed in public administration—such as racial inequity, resource scarcity, and a lack of public trust—and these complexities need to be confronted by public managers in the long-term (O’Flynn 2020; Sowa and Lu 2017; Sancino, Garavaglia, Sicilia, and Braga 2020).

As new issues emerged and entrenched challenges worsened, the question of how to make critical decisions was consistent. Public management scholarship has long promoted the benefit of performance management, a system of decision-making for public organizations that depends on data analysis and use, and it’s a promising tool for these unprecedented times (Hall 2021). As Dunlop, Ongaro, and Baker noted in their 2020 appeal to scholars, it is essential to study the analytical and learning capacities of public organizations during the pandemic so that we can make them stronger, more equitable, and better equipped to handle future threats.

For this research we compared use of performance management in core local government service departments within the state of North Carolina. This comparison allows for variation between jurisdictional use of performance management as well as functional departments within this one state case study.

Performance management during turbulence

Performance management “generates performance information through strategic planning and performance measurement routines and that connects this information to decision venues, where, ideally, the information influences a range of possible decisions” (Moynihan, 2008: 5). In other words, performance management is a system that allows public managers to analyze quantitative and qualitative performance information to make evidence-based decisions that will improve overall performance (Poister, Aristigueta, and Hall 2014). Behn (2003), an often-referenced scholar, outlines the multiple reasons to measure performance. His typology, consisting of eight relatively accepted managerial uses of performance information = that includes: evaluate, control, budget, motivate, promote, celebrate, learn, and improve.

The literature identifies two competing theoretical underpinnings about the effects of performance management on organizational performance. Traditionally, these systems were implemented as mechanisms to hold public employees accountable (Poister, Pasha, and Edwards 2013). Following the expectations of principal agent theory, public employees were seen as self-interested agent who look for ways to steal, shirk, and subvert the priorities of the departmental and political leadership (Dilulio Jr 1994). The accountability-based application of performance led to various negative unintended consequences including data manipulation, cream skimming, and inequitable delivery of public services (Pasha, Kroll, and Ash 2021; Radin 2006).

As a result, public organizations are increasingly moving toward a motivation-based application of performance management system (Pasha 2018). Goal-setting theory (Wright 2003) finds that clear and challenging goals motivate employees by communicating organizational values and expectations. The feedback through performance metric help employees learn about their progress and make evidence-based decisions.

Most literature on performance management focuses on the use of performance information and its impact (Kroll 2015; Gerrish 2016; Moynihan and Pandey 2010). These studies have produced contradicting evidence surrounding the impact of implementing a performance management system. Studies such as Sun and Van Ryzin (2014), Poister, Pasha, and Edrward (2013), and Pasha (2018) show a positive impact of performance management in education, public transportation, and criminal justice. Gerrish (2016) and Koning and Heinrich (2013) demonstrate mixed and null effects in child support and social services. Other such as Dias and Maynard-Moody (2007), Soss, Fording, and Schram (2011), and Pasha, Kroll, and Ash (2021) show negative impact of these systems on equity. Still Kroll (2017) find evidence for contingency factors such as stakeholder influence, and the presence of protective institutions to mitigate negative impacts.

More recently, scholars including Ammons (2020) and Kroll and Vogel (2021) have stressed the importance of multiple organizational stakeholders in implementing effective systems. Indeed, performance management systems are most effective when they are used and appreciated by multiple organizational stakeholders including administrative executives (such as the manager), strategic policy decision makers (the board), and the operating core (those within the department). For performance measurement to move to performance management it needs to be engaged, active, and accepted by a range of critical organizational stakeholders to be truly embedded in institutional decision making (Ammons 2020; Pasha 2018; Yang and Pandey 2009).

Notwithstanding the conflicting evidence, the adoption and use of performance management systems continue to increase (Pollitt 2016). Still, relatively little is known about how these systems respond during a crisis. The ongoing pandemic has renewed scholarly interest in performance management, which can play a central role in helping public managers make better decisions in uncertain situations (George, Verschuere, Wayenberg, and Zaki 2020; Hall 2021; Mitchell et al., 2021).

In their discussion of the COVID-19 pandemic, George, Verschuere, Wayenberg, and Zaki (2020) offer comparative data analysis as a means of making well-informed decisions while dealing with the crisis. Along similar lines, Hall (2021) emphasizes the critical need for data measurement in turbulent environments to gain a better understanding of the situation and to make evidence-based decisions. But as Yang (2020) points out, it is difficult to arrive at data-informed decisions when resources are depleted, competing values must be balanced, and time is constrained. He proposes that other factors, such as “trained intuition” and “reasonableness,” be employed alongside data when making decisions. Research in other disciplines such as accounting, nonprofits, and healthcare has also put a greater emphasis on performance management as a tool to mitigate the impact that COVID-19 had on services (e.g., Kober and Thambar 2021; Huber, Gerhardt and Reilley 2021).

Empirical evidence on the use of performance management systems during a crisis is scant, contradictory, and primarily concerned with the 2008–2009 Great Recession. Jimenez (2013), for example, found that these systems only increased the perceived ability of public managers to reduce deficits, not the actual deficits. In contrast, Pasha and Poister (2017, 2019) found that public transit agencies collected more data during the Great Recession.

Previous scholarship has also presented inconsistent conclusions about the long-term effect of crisis situations on organizations. One view holds that significant events, such as the COVID-19 pandemic, inevitably lead to profound transformation (Anderson, Branch, Cooper, and Dulvy 2017; Wind, Rijkeboer, Andersson and Riper 2020). Mostly driven by external forces, such events pave the way for large-scale changes by breaking down political consensus, generating extraordinary pressure on resources, or altering the social and physical landscape of an organization (Ansell, Sørensen, and Torfing 2020; Peters, Pierre, and Randma-Liiv 2011).

Crisis often causes latent conflicts to emerge, opening the field to power struggles among various stakeholders (Peters, Pierre, and Randma-Liiv 2011), who are emboldened to push for changes that would have been unthinkable during more stable times. As noted by Ansell, Sørensen, and Torfing (2020), the kinds of turbulence caused by “surprising, inconsistent, unpredictable, and uncertain events,” like the COVID-19 pandemic, “persistently disrupt our society and challenge the public sector” (p. 949).

In contrast, institutional and path dependent perspectives expect far less transition or change. According to this view, turbulence restricts broad changes for two primary reasons (Peters, Pierre, and Randma-Liiv 2011; Thelen 2000). First, a crisis often accompanies (or causes) a severe pressure on resources, which constrains the organization’s ability to invest in fundamental changes to organizational structure, personnel, and practices (Goodin 1996; Levine 1979). Second, fundamental changes to the organization are likely to cause stress, anxiety, and confusion to employees, intensifying their resistance to change and guaranteeing its failure (Bovey and Hede 2001; Bordia et al., 2006). Proponents of this view believe that decision-making methods and associated data use do not undergo significant changes in a crisis (Thelen 2000).

Indeed, public institutions are inherently conservative arrangements of nested rules, exclusive membership, and hierarchical authority, and they tend to prefer sticking to and maintaining the status quo (Hodgson 2006; North 1990). That status quo is further supported by the unyielding power dynamic among employees, clients, and elected officials that resists fundamental shifts (Pierson 2000). In support of this assertion, Kim and Chen (2020) found that local governments in New York State repeatedly employed the same cutback strategies in dealing with multiple recession events over decades. But the question that remains unanswered is if an ongoing pandemic might be enough to change the status quo altogether.

The literature thus presents competing expectations of how performance management could change because of the pandemic. Either the COVID-19 pandemic will transform how performance management is used (e.g., Ansell, Sørensen, and Torfing 2020) or the status quo will continue (Pierson 2000). Our research attempts to elaborate on questions of performance information use during the COVID-19 pandemic by focusing on the local government setting. We seek to explore whether and to what extent municipalities changed performance management use during the COVID-19 pandemic and what factors impacted such changes.

Survey measures

The survey was divided into sections that evaluated analysis, use, stakeholders, demand, and formal decision-making. Multiple survey questions measured performance information analysis and were developed by drawing on previous work by Pasha and Poister (2017) and Kroll (2015). Performance information use measures were developed by drawing on Behn’s 2003 typology of performance information objectives: to motivate, evaluate, control (hold accountable), budget, motivate, promote, celebrate, learn, and improve.

Recognizing the reality public managers faced in responding to the COVID-19 crisis the focus on promoting and celebrating were excluded from the survey. Given the timing of the survey and the extreme stress local governments were facing the researchers determined that implying that these directors should be focused on promoting and celebrating in the midst of a devastating public health crisis could be seen as insensitive.

A demand measure was used to determine the extent to which service demand changed during the pandemic. Finally, the level of formalized strategy pre-pandemic (2019) and during the pandemic (2020) years was included (Pasha and Poister 2017). The survey captures the use of performance information for decision-making by department heads in 2019 (before the pandemic), measured on a 7-point Likert scale, as well as how much that practice has changed (whether it increased or decreased) during the pandemic, measured on an 11-point slider scale (See Appendix A for survey items). This approach helps us explore change with respect to the previous year’s behavior, allowing respondents to cognitively focus on the change in level with respect to the base year. Given the perception-based nature of this data, this approach allows for greater variation and prevents the potential of anchoring level use in which respondents default to reporting the same response patterns as they had indicated for the past year.

This approach allows us to ascertain the perception of performance management use before the pandemic (i.e., the initial condition) and the perception of how it changed during the pandemic (Duit 2007; Pierson 2011). By anchoring the survey in past use, we could draw conclusions regarding relative (rather than absolute) changes from 2019 to 2020. All of the data is subjective and depends on self-reporting by each responding department director.

We recognize social desirability as a concern for survey responses. The survey specially asks for change in use rather than rate of use for the two years to avoid anchoring bias or halo effect between years. This perception of use by managers was important to consider given the interest in performance management beyond performance measurement. While we recognize the concern of bias, variation in results including those reporting limited change lessen that this effect is prominent in the data.

Sample

We collected survey data from six municipal departments (budgeting, human resources, inspections, parks, planning, and public works) in North Carolina cities and towns with populations greater than 10,000 (90 municipalities qualified for inclusion based on size). The specific departments represent fundamental core services across surveyed municipalities. They also offer a mix of internal and external-facing functions. The population size of the municipalities increased the probability that their workforces would be large enough to have dedicated staff within these departments.

Email addresses for department heads were assembled from municipal websites. The survey was administered to 472 department directors. Some municipalities lacked the departments within their organizations completely, especially for the smaller jurisdictions. Municipalities that contracted services to the county or had consolidated departments were excluded. Some websites did not provide contact information for departmental director resulting in them being excluded from the final sample. We received 103 responses (21.82% response rate). Table 1 shows the distributions of respondents by department.

OPEN IN VIEWER

Table 1.

Survey respondents.

Department	Number
Budgeting	21
Human Resources	19
Inspections	13
Parks	14
Planning	21
Public Works	15
Total	103

Analysis

To understand what changes municipalities made to the level of performance management use during the COVID-19 pandemic, we used descriptive statistics to compare performance management analysis and use in 2019 (pre-pandemic) and 2020 (pandemic) across the full sample. Next, we examined the changes by department type to determine whether there was a variation in how different local government departments changed their performance management practices during the crisis.

An Ordinary Least Squares (OLS) regression analysis was used to determine the changes in performance management practices during the pandemic. This helped address the second research question of this study, which examines whether pre-pandemic performance management use affected use levels during the pandemic. Equation (1) represents the estimation model.

X i represents the matrix of two control variables: the extent to which the department adheres to formal decision-making and changes in service demand during the pandemic. Given the small size of our sample we were limited on the number of control variables. We ran models that included organizational size and tenure of respondents, but they were not statistically significant. We were unable to determine political stability. ε i represents the unobserved random error, clustered by city.

Performance information analysis

Respondents indicated an increase in analysis of performance information during the pandemic (2020) compared to 2019 for all measured areas except as it relates to conducting comparisons with other units (benchmarking), which showed a decrease (see Figure 1). Since the 2020 response variables denote a measure of ‘change’ from the base year, a negative value for the dependent variable means a decrease in relation to the base year (2019), whereas a positive value means an increase.

OPEN IN VIEWER

Figure 1.

Change in Performance Information Analysis During the Pandemic.

Directors reported that even before the pandemic (2019), their departments engaged in benchmarking less than the other (internally focused) forms of performance information analysis—including monitoring, comparison with targets, and comparison with past performance—and the use rate declined further in 2020. It may be that if the practice and culture of using data drawn from comparing to external peers is not established before a crisis, the time-consuming process of developing benchmarks, collecting data, and properly analyzing the results are less likely to be initiated during a crisis. It’s also likely that individual department needs and expectations were changing so rapidly, especially in the early stages of the pandemic, that determining and stabilizing internal practices was a challenge let alone establishing comparison with external jurisdiction who were also in flux.

Municipal departments reported an increased use of performance information during the pandemic (2020) to monitor progress and compare it with internal targets and past performance. This increase is understandable since performance information gives public managers a better idea about the current situation, how far they are from achieving their goals, and how much they have deviated from the previous performance (Bidya 2009; Behn 2003; Grant 2003) — performance information that likely became even more important during the pandemic. As jurisdictions struggled to determine how to respond in uncertain times internal analysis could help to inform choices and support hard decisions.

Though the usage rates vary substantially, the trend is consistent across department types, except for budgeting departments (see Figure 2). In general, the various departments engaged in increased levels of performance information analysis as it related to monitoring performance information and comparing it with targets and past performance. The budgeting and finance departments, however, showed the smallest increase in performance information analysis and the biggest decrease in comparison with other measures during the pandemic. One possible reason for this is that, despite many budget departments facing substantially decreased budgets and enhanced workload to have to revise budget projections and associated communications, the overall budgeting process remained stable.

OPEN IN VIEWER

Figure 2.

Change in Performance Information Analysis During the Pandemic by Department.

Human resources was the only department that increased the use of benchmarking. Given that local governments were under tremendous pressure to figure out how to manage a workforce in a crisis while moving large numbers of employees to remote work (Delfino and Van der Kolk 2021), it makes sense that human resources departments wanted to know how their colleagues in other municipalities were responding.

Performance information use

Performance information use refers to the practice of making decisions and taking action based on the information generated by a performance measurement system (Moynihan 2008; Poister, Aristigueta, and Hall 2014). Overall, the surveyed municipalities reported an increased use of performance information focused on motivation, evaluation, budgeting, accountability, and learning during the pandemic (2020) (see Figure 3). Not surprisingly, the use of performance information for the purposes of motivation, evaluation, and accountability increased less than for budget and learning. With economic challenges hitting municipalities hard, budgets had to be substantially revised. The biggest increase was seen in the use of performance information for learning. The immense changes caused by COVID-19 required constant learning—perhaps one of the more encouraging consequences of the pandemic.

OPEN IN VIEWER

Figure 3.

Change in Performance Information Use During the Pandemic.

Across departments, the pattern of enhanced use remained consistent except for budgeting departments (see Figure 4), which reported a lower-than-average use of performance information for motivation both before and during the pandemic. Given the economic downswing that accompanied the pandemic, many budget and finance offices were scrambling to redo their budgets and make massive, unplanned cuts. Their priority was financial data—not data for managerially efforts. Though this lack performance information designed and collected to inform and improve managerial efforts on motivation is an area for improvement for these offices that will need to not only motivate but worry about retention of talented public servants.

OPEN IN VIEWER

Figure 4.

Change in Performance Information Use During the Pandemic by Department.

The departments of inspections and permits used performance information for employee motivation with the same intensity as the prior year but decreased their use of performance information focused on holding employees accountable. Given the changing nature of work during COVID-19 (especially early in 2020), accountability measures may not have been as viable as in previous years—with industries putting projects on hold plus the problem of safely conducting in-person inspections during a pandemic, many inspectors simply couldn’t complete their assignments. In the pre-pandemic period, the departments of planning and economic development made limited use of performance information for employee motivation or budgeting decisions, but they dramatically increased their use of it during the pandemic (2020).

Stakeholder interest in performance information

Performance information usage rates are an important part of the picture of performance management within local governments, and it’s also important to better understand who is tracking, using, and interested in the data. Department heads reported an increased use of performance information by line department staff, city managers or management teams, and elected boards in 2020 compared to the previous year (see Figure 5). While all groups saw an increase, the levels before and during the pandemic varied. Respondents reported the greatest increase for this survey item: “Our city manager/management team regularly monitors our performance measures.”

OPEN IN VIEWER

Figure 5.

Change in Performance Information Use by Stakeholders During the Pandemic.

Performance information use at the staff level (anyone below the director) allows employees to better understand where the organization is headed and whether their strategies are working. For this analysis, we looked at the question of staff use by asking directors the question: “My staff frequently uses performance measures in proposals and requests they make to me.” Directors reported little staff use of performance information for this purpose in 2019 and only a small increase in 2020.

As noted above, the department heads indicated that their city managers monitored performance information more frequently during the crisis (2020). The increase could have been driven by their need to understand where the department was headed and what decisions had to be made as well as to respond to board and citizen requests for information. It also could have been a result of city managers’ need to make complex decisions across departments that had competing service needs.

The directors indicated that board interest in performance information during the pandemic increased, likely a consequence of board members’ need to respond to an array of concerns expressed by citizens and a desire to feel an increased level of understanding during very uncertain times.

The directors of all departments except budget departments indicated that staff use of performance measures in proposals and requests increased at varying levels during the pandemic. In 2019, budget offices were already infrequent users of performance information for this purpose, and they used it even less in 2020, likely because of their intense focus on financial matters and standardized budget processes. The budget directors in the survey reported that board interest in their performance information also declined, though the monitoring of their performance information by city managers increased in 2020.

The political pressure on elected officials during the pandemic was high. As thorny issues of mask mandates and office, business, and school closings took center stage, some departments invariably shifted away from internal data. Directors of human resources, inspections, and, to a lesser degree, parks departments did acknowledge increased interest from board members in their performance information. All departments with the exception of parks (most parks were closed during this time) saw their city managers increasing the extent to which they monitored performance data.

Overall, with few exceptions, the directors indicated that other stakeholders showed an increased interest in and use of performance information during the pandemic (2020) compared to the previous year. As hard decisions were being made in all departments, directors of human resources consistently indicated the greatest increase, which was not unexpected given the workforce changes in 2020 and the need to design and monitor new approaches to work.

Regression analysis

Consistently, municipal departments reported increased levels of performance information analysis and use during the pandemic (2020) compared to the previous year (see Figures 1, 2, 3, 4, 5 and 6). What factors could explain these changes? To address this question, we examined OLS regressions estimates with a focus on the extent to which prior performance management practices may help to explain pandemic practices.

OPEN IN VIEWER

Figure 6.

Change in Performance Information Use by Stakeholders During the Pandemic by Department.

Table 2 shows the relationship between reliance on performance management practices before and during the pandemic. This analysis was done for both use and analysis measures to provide a more complete view of changes to performance management practices. Results indicate that one additional unit of pre-pandemic performance monitoring is associated with a 0.320 unit increase in the activity during the pandemic. Similarly, one additional unit of comparison with targets and other agencies before the pandemic is associated with 0.361 and 0.753 unit increases in these practices during the pandemic, respectively. Goals setting, identification of appropriate comparison groups, and collection all require careful analysis and planning. When such systems are in place it appears that they are being leveraged by the departments that have invested in them while those that have not are less likely to divert limited resources to developing these. While the trend for a number of years has been increased adoption and use of performance management systems it appears that the pandemic served to limit this trend as it did with many other initiatives and municipal strategic priorities.

OPEN IN VIEWER

Table 2.

OLS Regressions Estimating the Relationship between Performance Management Practices prior to and during the COVID-19 Pandemic.

	Monitor	Compare w/Target	Compare w/Others	Compare w/Past	Motivate	Evaluate	Budget	Hold accountable	Learn	Staff use	City manager use	Board use
	(1)	(2)	(3)	(4)	(5)	(6)	(7)	(8)	(9)	(10)	(11)	(12)
Pre-Pandemic Status	0.320*	0.361**	0.753***	0.181	0.647**	0.624**	0.462**	0.548**	0.524**	0.716***	0.789***	0.922***
	(0.168)	(0.142)	(0.170)	(0.196)	(0.248)	(0.247)	(0.214)	(0.239)	(0.210)	(0.177)	(0.191)	(0.189)
Formal Strategy	−0.174	−0.061	−0.484	−0.169	0.073	0.195	0.313	0.254	0.361	0.408	0.284	0.323
	(0.206)	(0.207)	(0.291)	(0.266)	(0.236)	(0.237)	(0.286)	(0.261)	(0.262)	(0.295)	(0.217)	(0.303)
Service Demand	−0.187	−0.089	−0.081	−0.121	0.156	0.166	0.471*	0.221	0.309	−0.073	0.130	−0.139
	(0.190)	(0.173)	(0.202)	(0.233)	(0.225)	(0.211)	(0.255)	(0.216)	(0.276)	(0.230)	(0.231)	(0.254)
Budget Dummy	−1.130***	−0.879**	−0.849	−0.968	−0.321	−1.181**	0.488	−0.505	−0.409	0.216	−0.066	−0.018
	(0.328)	(0.376)	(0.714)	(0.577)	(0.585)	(0.545)	(0.639)	(0.538)	(0.623)	(0.482)	(0.517)	(0.702)
Constant	1.021***	0.865**	0.049	1.227**	−0.182	−0.160	−0.464	−0.236	−0.053	0.094	−0.216	−0.072
	(0.358)	(0.375)	(0.415)	(0.497)	(0.544)	(0.528)	(0.579)	(0.511)	(0.605)	(0.529)	(0.544)	(0.599)
Obs.	69	58	56	53	53	50	52	49	48	49	50	37
R-squared	0.147	0.185	0.256	0.072	0.282	0.385	0.237	0.302	0.300	0.373	0.411	0.529

Municipal departments that used performance information to motivate employees before the pandemic were likely to increase this practice by 0.647 units during the pandemic. One higher unit of pre-pandemic use of performance information to evaluate, budget, hold accountable, and learn is associated with a 0.624, 0.462, 0.548, and 0.524 unit increase during the pandemic, respectively. One additional unit of pre-pandemic use of performance information by staff is associated with a 0.716-unit increase, whereas one additional unit of the perceived city manager and board use is associated with a 0.789 and 0.922 unit increase each during the pandemic. Organizations that have done the work to consider the managerial purposes, associated performance measures, collection and use are more likely to continue to deploy these measures and use them in times of intensified decision making as was marked by managing in the midst of the pandemic.

Additionally, these findings support scholarship (Ammons 2020; Kroll and Vogel 2021) that highlight the importance of increased interest by stakeholders of performance management systems. If stakeholders are practiced in using and relying on performance information the probability of use of such information in times of crisis or confusion will increase. A crisis is not a time to learn how to request, apply, and interpret information that is not part of the established decision-making models.