Workarounds and shadow IT – balancing innovation and risk

Introduction

For over five decades I have walked through the door of new client to face an initial challenge of distinguishing fact from fiction. The fiction is that employees are using processes which have been designed and optimized to ensure maximum productivity. The fact is that the client has no knowledge of the extent to which employees have developed workarounds to what they regard as poorly designed applications.

The difference between the fiction and the fact can result in a significant increase in corporate risk and employee dissatisfaction.

‘Cut and paste’ technology was developed by Larry Tesler at Xerox PARC (Long 2023) in the mid-1970s. It is now used extensively not just to modify a document we are working on but also to extract and then insert content from another document or application. It saves time re-keying the content and is an example of a workaround though we probably do not consider this as the underlying concept at the time of doing so.

The scale of ‘cut and paste’ is now being changed dramatically with the advent of AI Generated Content (AIGC) applications such as ChatGPT. A survey in 2023 (Fishbowl 2023) indicated that 70% of workers were using ChatGPT without the knowledge of their managers. Another example of a workaround is the use of shadow IT applications which are being used without the knowledge, permission and support of a corporate IT department. (BMC 2022)

Workarounds and shadow IT can be of benefit to the individual employee and the organisation through being a source of innovation for process and application development. They can also substantially increase corporate risk and information security. However, most of the research into workarounds has explored the use of workarounds in enterprise data management applications and to a lesser extent workarounds in clinical applications. Little attention has been paid to understanding the use of workarounds in the digital workplace and their potential impact on the level of trust in enterprise information.

Enterprise complexity

In 1974 Lehman presented his Laws of Software Evolution (Lehman 2007) in his inaugural professorship lecture at Imperial College, London, where from 1972 to 2002 he was a Professor and Head of the Computing Department. The first two Laws were

• Software systems must be continuously changed to adapt to the environment.

There is no accepted measure of ‘complexity’, probably on the basis that it is (like search relevance) a personal and subjective assessment. It is certainly the case that business requirements are constantly changing, especially since the Covid pandemic, and these changes need to be matched by enterprise-wide systems. The marketing of these systems is often supported by schematic diagrams which attempt to illustrate the interconnectivity of individual enterprise systems, often accompanied by the proposition that all that is required is an Enterprise Application Integration software suite.

“Enterprise application integration (EAI) is the task of uniting the databases and workflows associated with business applications to ensure that the business uses the information consistently and that changes to core business data made by one application are correctly reflected in others.” (Wikipedia 2023).

In principle this would seem to be a positive direction to improve the impact of IT on business performance. However, descriptions and schematics of EAI software omit any reference to the ‘end user’. Over the last few years, the ‘User Experience’ (UX) has been recognized as being of considerable importance in ensuring that customers can make effective use of web sites. There seems to be no recognition in the implementation of enterprise systems of the ‘user experience’ of employees who are faced with using increasingly complex applications.

Much of the credit for improving the usability of web applications lies with Don Norman and Jakob Nielsen, both of whom started working on user experience topics, coming together in 1998 to establish the Nielsen Norman Group (Nielsen Norman Group 2023). Employees are now well aware of what good web usability looks like and are inevitably critical of enterprise applications that they judge to have poor usability, especially when it inhibits their ability to achieve personal and business objectives.

There is now a substantial amount of research that employees, faced with a level of complexity that they find stressful or unproductive, develop workarounds to corporate systems and and make use of shadow IT applications. The existence of these workarounds and shadow IT use is invariably invisible, even to their immediate managers, because of concerns that they will be admonished and even penalized for not conforming to the prescribed way of using these systems.

The outcome is to increase the risk profile of the organisation and to increase the technical debt it operates with, especially where these workarounds are adopted in knowledge-based processes.

A short history of processes and workarounds

The starting point is the establishment of the concept of a business process dates back to 1368 and an Act of Parliament which set out a legal process which had a number of defined steps which had to be worked through sequentially to the conclusion of the court case.

Exactly how and when the term ‘workaround’ was first used is lost in time but certainly it was in common usage in the US aerospace industry in the early 1960s, reaching a pinnacle of public awareness in the way in which NASA managed to bring the damaged Apollo 13 space craft safely back to earth. (Kranz 2000)

The adoption of the concept from the perspective of academic research dates back to 1984 and the work of Gasser (1986) on how ‘working around’ (interestingly he did not use ‘workaround’ in his paper) the challenges of complex enterprise IT applications needed to be recognized and managed. The fact that there are currently over 750 citations to Gasser’s paper is a testament to his early and novel appreciation of problems that users of enterprise IT applications would be faced with and the scale of subsequent research.

Fitness to specification, but not to purpose

It was in the early 1980s that the way in which office work would be changed by the advent of personal computers was being considered. Xerox PARC was at the forefront of this research, with Suchman (1983) and Gerson and Star (1986) raising concerns about the potential gaps between fitness to specification and fitness to purpose.

The specification for an enterprise system will set out what the system (whether new or an upgrade) needs to do in terms of its functions (‘create a list of out-of-stock items by customer) based on interviews with managers and perhaps some employees who are using the existing system. Non-functional requirements (primarily related to adequate usability) are a much greater challenge because employees will have a different view of usability depending on

• Their knowledge of the business, its products and its procedures.

A very helpful categorization of core issues of usability has been developed by (Hertzum 2010), setting out six ‘images’ of usability which illustrate the many dimensions of system usability, and the potential points of dissatisfaction, frustration and failure.

• Universal usability—usability entails embracing the challenge of making systems for everybody to use.

An important element of cultural usability is that there are often multiple languages being used within the enterprise but error messages are invariably in English and may not be usefully understood by employees.

Comparatively little research was carried out into enterprise application implementation and use in the period between 1984 and around 2010. By then it was becoming painfully obvious that implementing enterprise-wide applications (in particular enterprise resource planning applications) was a far from straightforward task. One summary of the situation referred to ‘clumsy implementations’ (Newell 2007) and that is still a fair description.

Where there is a gap between functional and non-functional requirements then employees may well have no option but to resort to workarounds. In the example above (a list of out-of-stick items by customer) an employee may find it easier to download the data from the application and use an Excel spreadsheet (as shadow IT) to compile the listing, perhaps then adding this list to the system for use further down the process line.

Hospitals go digital

Enterprise-wide applications, notably Enterprise Resource Planning (ERP) applications, date back to the mid-1960s so there is usually a substantial amount of experience in an organisation of the challenges of specifying, implementing and managing these applications. In parallel a few major US hospitals (notably the Mayo Clinic) started to develop Electronic Health Record (EHR) systems. (Elliott 2022)

In 2004, President George W. Bush created the Office of the National Coordinator for Health Information Technology, outlining plans to ensure that most Americans had electronic health records within the following 10 years. Then in 2009, President Obama incorporated EHR into his American Recovery and Reinvestment Act, as part of the Health Information Technology for Economic and Clinical Health Act (HITECH). This provided additional funding to health care providers that met particular criteria, which involved using EHR and meeting technological requirements by 2014. The US went from 10% of hospitals using an EHR system in 2008 to over 80% in 2015.

EHR applications are now ubiquitous in the USA and are widely adopted globally. However implementation issues have been challenging because hospitals, especially outside of the USA, were faced with moving from paper-based records to digital patient records.

In many respects EIS and EHR applications gave rise to similar problems but some important differences were starting to emerge. Among these were the much wider integration of text (in the form of notes on treatments and outcomes) and the role of nurses in particular as a source of innovation in not only driving application development but being aware of the implications on treatment outcomes. (Bartelheimer 2023)

This contrasts with the situation in enterprise applications where there is little involvement in employee-supported development of systems and a sense that workarounds should not be tolerated. The enterprise focus is on conformance to corporate policies and especially on improving productivity.

There is also a much greater need for inter-operatibility with other primary and secondary clinical systems as patients move between clinics and hospitals. This has resulted in the development and compliance with global interoperability standards. (Ignite Data 2022)

Interestingly the two communities seem to have no opportunity to learn from each other apart from the academic research literature which senior IT managers in enterprises are unlikely to have access to or an incentive to read.

Definitions and categorisations

A substantial amount of research has been undertaken over the last 50 years with the objective of defining the characteristics of a workaround and developing categorisations of workarounds as an aid to their detection and management. (Ejnefjäll 2023)

There is general agreement is that a workaround is the use of an application that is supported by a corporate IT department, whereas shadow IT (often written as Shadow IT) is the use of unsupported and unapproved IT applications which may either have been acquired and supported by a department or are the personal IT applications of employees, notably the use of smart phones. (Hulsebosch 2023; Klotz 2019; Rakovic 2020; De Vargas Pinto 2022; Soffer 2023).

Arguably shadow IT is a workaround but a workaround might not involve shadow IT.

Two important pieces of research were being undertaken in the early 2010s by and by Van der Shaft-Bartis (2013) and Alter (2014). Alter was focusing on a definition for workarounds and whether the definitions could result in a classification of types of workaround that would enable IT managers to manage them.

Van der Shaft-Bartis was exploring approaches to the definition of workarounds but of perhaps greater importance was an analysis of the ways in with workarounds could be discovered, given that employees who had developed workarounds had incentives not to disclose them outside of a small group of colleagues. Among the discovery techniques was that of ethnography, which used carefully designed interviews with users to explore the extent and use of workarounds.

Making the invisible visible

It is in the nature of both workarounds and shadow IT that the incidence in the organisation will be unknown. This is a particular problem in clinical systems because of the risk to patient treatment outcomes. (Blijleven 2017; Tucker 2019; Lee 2021) Any survey of a company is unlikely to arrive at even an approximate level of adoption.

In the many case studies that have been undertaken, the choice of the employees to interview would have been made by the company as being representative of core business processes. It could be argued that this is close to a random sample and that if the interview programme reveals a substantial incidence of workarounds from a small group of employees then that could be an indication that workarounds are likely to be endemic in the organisation.

A considerable amount of investment is now being made in Business Process Management (BPM) and Process Mining (PM) applications which track the course of processes in terms of chronology and keystrokes with the promise that the aggregated data will enable to the enterprise to identify workarounds from differences in both. (Bade 2022; Beerepoot 2021; Blijleven 2017; Outmazgin 2016; Weinzierl 2022)

The challenge for both external academics and consultants is to arrive at a balance between quantitative assessments of the existence and nature of workarounds using some form of data logging and the use of qualitative ethnographic methods. (Ducheneaut 2010; Morike 2022)

The use of ethnographic methods brings with it the possibility of bias. The nature and extent of this bias may be different when the work is undertaken by an external consultancy compared to an internal team. To further complicate the situation, there could be differences between a specialist external consultant and an academic team with limited prior experience of these methods. Using an internal team to uncover workaround issues will inevitably result in the outcomes being challenged by both employees and IT manager.

A further challenge with understanding the potential impact of workarounds is that the true impact may be some distance downstream from the initial workaround. (Drum 2015; Drum 2016; Drum 2017; Persteiner 2018). The employee may have no knowledge or understanding of the potential impact of the workaround they have created on subsequent steps in the process.

In many instances there will be benefits from using both quantitative and qualitative approaches and then using the disciplines of mixed method research to integrate the outcomes.

A common thread through both is a concern about data privacy. This is of course a major concern in the clinical sector but is also an issue in the enterprise sector around the identification of specific employees being tracked through data logging. (Bade 2022) This is not just about GDPR conformance as about the level of proof that a data logging application can give about the activities of an individual employee and how this information might be used in assessing the performance and career development of the employee.

Catalysts for workaround

The published research literature on workarounds is extensive, with probably in excess of 1000 publications. However, the research process invariably on quite small scale on-site surveys that rarely consider the reasons why employees adopt workarounds. The core reason for this situation is that employees may be very unwilling to disclose the reasons because of a concern that the organisation will view the workarounds as a lack of commitment to corporate policies. There are also indications that managers tend not to look to far into the existence of workarounds in case the situation rebounds on them. (Röder 2014)

Workarounds are being developed by individual employees to enable them to achieve acceptable levels of productivity without the stress of working with an approved and internally supported application which is difficult to use. (Marsh 2022)

Only comparatively recently has the issue of psychological safety been considered as a progenitor of workarounds. Psychological safety is the confidence that an employee will not be punished or humiliated for speaking up with ideas, questions, concerns, or mistakes. It is also present in the expectation held by members of a team that colleagues will not embarrass, reject, or punish them for sharing ideas, taking risks, or soliciting feedback.

An influential paper on these issues was published in 2003 (Baer and Friese 2003) in which they argued that process innovations, defined as deliberate and new organizational attempts to change production and service processes, need to be accompanied by climates that complement the adoption and implementation of such innovations. In this context process innovations are, in effect, workarounds.

The paper goes to the heart of the matter when it comes to taking advantage of workarounds, and shadow IT, to improve process performance and personal recognition. As of the time of writing this book the paper had been cited 2344 times, which indicates both the value of this research and the scale of subsequent research. A recent paper on this topic (Edmonson and Bransby 2023) provides a comprehensive overview.

Accommodating neurodivergent employees

The concept of neurodiversity dates back to the late 1990s as a way of describing a wide range of cognitive conditions such as autism, dyslexia and Attention Deficit Hyperactivity

Disorder (ADHD). These are all ‘spectrum’ conditions and are often very difficult for both the person and the clinician/psychologist to identify. There are no remedial treatments for any of these conditions. Employees with these conditions find workarounds in both daily life and in the workplace in order to achieve as good a personal outcome as possible.

We can gauge to a limited extent the problems faced by employees who are blind, colour-blind or have conditions that affect the way in which they can use a keyboard, touch screen or a mouse. There is no way in which we can appreciate the challenges faced by employees with a neurodivergent condition.

Where an employee resorts to a workaround to accommodate their particular neurodivergent condition (or indeed, conditions) they may be even more reluctant to disclose the workaround they have adopted in case it leads to a more general discussion about the impact of their neurodivergence on their ability to undertake the roles and responsibilities that their position requires.

There is very little published research on the range of workarounds adopted specifically by employees with a neurodivergent condition in an enterprise setting. (Das 2021) examines the issues arising in the case of neurodivergent employees working remotely and this gives an indication of the issues they would experience in an on-site situation.

Knowledge work processes

The research into workarounds and shadow IT has focused almost entirely on what might be termed data management processes, such as order, supply and invoice processing. Here the processes are well defined and usually either linear or with well-documented conditional branching. Furthermore there are usually validity checking routines to ensure that (for example) a customer number always contains the same number of characters.

In an office/document environment there can be multiple contributors to a document in a mix of parallel and linear paths. This is especially the case when contributions have been made from multiple locations in a multi-national business. Often intermediaries are involved in managing the flows of information. A paper (Brooks 2018) explores these complex issues in some detail with quotes from participants.

The important distinction is that it is highly likely that a document is prepared, often by multiple contributors in different roles, departments and locations for a reader to make a decision, and that decision inevitably carries with it a degree of risk. A useful illustration of a decision process is that presented by Citroen (2011) in a study of how senior executives collect the information they need to make strategic decisions. The multiplicity of actors and processes involved in the preparation of a document makes it very difficult to identify where workarounds have been undertaken.

A core issue with knowledge work is the requirement to be confident in the quality of information that is being used in decision making. One of the few studies of the impact of workarounds on information quality was undertaken by Laumer (2017) who highlighted that very little research has been carried out on workarounds in content management systems. The organisation was a financial service provider with approximately 900 employees. The organisation had introduced a web-based enterprise content management (ECM) system to support organisational processes and employees’ work routines, providing information not covered by the core IS (e.g., core banking system)but required to support sales talks and other work routines.

Among the workarounds that emerged from the research were

• Employees call experts by phone when they have any a question instead of searching for the information they need in the ECM system.

This list highlights the diversity of channels through with information and knowledge are transmitted in an organisation, Google Scholar listed 151 citations to this paper but on inspection relatively few are directly concerned with workarounds and are citing the paper because it provides a good overview of the lack of information management discipline in organisations. (Wibisono 2022; Willermark 2022; Wolf 2022)

Organisations are likely to be unaware of the scale of employees working around a problem by making contact with an ‘expert’ as there is invariably no process to log an approach to an expert. The use of AIGC applications as shadow IT to obtain an ‘expert opinion’ on a topic may increasingly be adopted as a workaround.

Digital workplace

The concept of a digital workplace (White 2012, 2020) has changed significantly as an outcome of the Covid pandemic and the adoption of hybrid working. Where there is hybrid working the chances are that there will be hybrid processes involving workarounds and shadow IT. The extent of these workarounds may be increased by the need to extend a digital workplace to a customer or a supplier who may be running different systems. It may be mutually advantageous to use DropBox for a common file share and Zoom as a conferencing application even when the corporate policy is to use Microsoft Teams.

Crossing the firewall

Information workarounds inside an organisation will probably have little immediate impact outside the organisation. A notable exception of that assumption is the case of financial information, even if it is not for public circulation. (Drum 2015) has considered in some detail the issues that can arise in financial reporting where workarounds have resulted in some degree of corruption of the financial records of the organisation, records that will then be used by internal and in particular, external auditors, to assess the financial performance of the organisation.

Subsequent papers (Drum 2016; Drum 2017) take this framework further to assess the problems that organisations face in collecting and managing financial information, as this information will have to be forwarded to external auditors for validation.

The dark side of information

(Stone 2020) reviews the literature on information mismanagement and constructs a typology of misinformation that can be applied to analyse project planning and strategic planning processes to reduce the chances of failure that results from information mismanagement. One of the categories in their list of potential sources of what they denote as Dark Side Information Behaviour (DSIB) are system or process issues such as

• Information incompetence systems and processes do not deliver required information and the situation is tolerated.

There is no further analysis of the extent and impact of systems-related issues, and the concept of workarounds is not considered. Bawden (2009) also considers the issues of information overload and employee anxiety. The challenges of enterprise system implementation in multinational corporations are discussed in detail by Malaurent (2019).

Information management in a clinical setting

Ensuring that information collection and distribution in an organisation is not compromised by workarounds is of primary importance in clinical settings using electronic health care records. (Jylha 2016) considers incident reports relating to situations where information accuracy has been compromised. This paper does not explicitly include workarounds in the research and analysis but does illustrate the wide range of information-related issues that can arise. (Persson 2021)

This is also the case with a thesis by Elliott (Elliott 2022) but the value of the research lies in the direct quotes from clinicians and others managing patient notes under often significant time constraints. There are no specific references to workarounds but the interviews do indicate the pressures that clinical staff experience in managing patient records.

Risk management and technical debt

In the case of both enterprise and clinical settings there is a strong commitment to reducing risks. In the enterprise these risks are related to conformance to internal standards and policies (such as ISO27010 on information security) and to external audits for financial matters, as well as potentially an impact on corporate reputation. In a clinical situation patient wellbeing and positive treatment outcomes are monitored very carefully and reported to external agencies.

The problem for both environments is how the risks arising from invisible process workarounds and shadow IT can be quantified. This is especially the case with shadow IT which brings some substantial information security management implications – with workarounds this is less of a problem as the employee is using an approved application.

Technical debt represents the investment in application development which make no impact on the overall performance of the organisation. Upgrades to functionality may be costly to design and implement but the over-riding lack of commitment to usability means that employees will continue to find alternative ways of meeting their objectives.

Information management implications

Although written almost a decade ago the paper by Alter (Alter 2014) remains the definitive starting point for any consideration of the impact of workarounds on an organisation. Although there has been a substantial research effort over the last 10 years into identifying and categorizing the workarounds that employees have adopted it remains very difficult to identify what the issues are within an organisation that might catalyse workarounds and the use of shadow IT.

Data logging applications can provide evidence to indicate the likelihood of a workaround being used, but do not generate a solution at either an employee, role or department level. Ethnographic methods offer a deep dive into specific situations but are very labour-intensive, prone to bias and do not scale. The range of user testing carried out on enterprise web applications (such as intranets) seem rarely to be carried out on enterprise database systems, perhaps because of the difficulty of making changes to user interfaces after implementation.

There is also a lack of awareness of the principles of effective information management. Even in organisations with a commitment to product and service quality there are rarely information management policies for information quality, nor an overall information management strategy. Information is supposed to flow around an organisation but invariably it does not (White 2020) and remains located in silos and team repositories. That in itself is a significant challenge but the reality could be that the information is located in cloud-based shadow IT applications where there are no audit trails and the risks are not identified and managed.

Information managers should be aware of the possible adoption of workarounds and shadow IT and take a default view that they could be important leads to process innovation. There is no benefit in attempting to survey and audit workarounds and shadow IT as the outcome could be that they become even more invisible. The outcome will be the same if there are top-down edicts that explicitly or even implicitly create the impression that employees are at fault in adopting them.

The organisation should create a culture that encourages employees to discuss their workarounds and use of shadow IT with their managers as the basis for identifying the opportunities for innovation. Managers need to be aware of the potential benefits and risks of workarounds and ensure that there are appropriate communications channels for employees to surface their workarounds and forums where the benefits and risks can be considered without prejudice to the employee.