Abstract
This case is about a cyberattack on a software publisher where the intruders created and consumed a lot of cloud computing capacity in a very short period of time. It was strongly suspected the purpose of the intrusion was crypto mining. There were no observed traces of any other malicious activity. A professional forensic investigation was inconclusive because of lack of evidence – it did not find any proof of data compromise, nor did it prove there was no data theft. In such situation, the CEO and the leadership team had a tough decision to make. They might be contractually bound to disclose this event to their customers, but there was serious misalignment on the interpretation of contracts. The potential penalties of a wrong judgement were high, and so were the consequences of voluntarily disclosing the incident to the customers. The CEO must decide what to do next – to disclose or not, and manage the risks in either case.
Keywords
Prologue
It was 1st Nov 2024 5 PM Eastern Standard Time (EST) and a leadership meeting of Artys Inc. was in progress. Participants were joining from different parts of the globe – the Chief Executive Officer (CEO) John, Chief Legal Counsel Josh, and Chief Information Officer (CIO) Greg were present in the company’s New York corporate office; Senior Vice President (SVP) of customer services Mahesh and Chief Technology Officer (CTO) Shankar were joining remotely from Hyderabad office.
They had a difficult decision to make, and the stakes were extremely high, possibly the survival of the company itself. For the last few days, Artys had been dealing with the aftermath of an intrusion in its network. All participants had a summary prepared by the CIO in front of them and had to decide if, what, and how to communicate to their customers. A lot was riding on that decision, and a wrong step could be catastrophic. There was no time to lose, and the decision now rested with John.
The background
Artys was a software company with three products that were mainly used by financial service providers – a lending risk analysis solution, a customer loyalty management suite, and a customer service product. The company sold its products as licensed software and as Software as a Service (SaaS). For SaaS customers, Artys hosted all its systems on a leading public cloud.
Artys revenue in 2023 (all figures in USD ‘000).
Most of the technical staff were based in India, and most of the sales and customer success staff were located in the United States. Except for a helpdesk engineer for local US time zone support, the rest of Greg’s team was based in India.
The Information Technology (IT) setup was not very complex. Customers were hosted on a shared infrastructure that had a cluster of application, web, and database servers that delivered to a group of SaaS customers. The corporate applications were all cloud-based, including Office 365 and enterprise systems for Finance, HR, and sales teams.
The setup was located in the US, Europe, and Asia regions of the cloud for customer proximity. The networks were virtually segregated into internal, customer non-production, and customer production segments.
Cybersecurity had not been a priority for the company, and funds were hard to come by. Greg had been highlighting this and asking for higher budgets unsuccessfully for a long time. John viewed IT as a cost centre and frequently asked Greg to optimize. Moreover, as he said often “Greg, we can’t afford a grand setup with our size!” The fact that they were hosted on the cloud instilled a sense of security and confidence in John. He had heard that public clouds were very secure.
Greg, though he was the CIO, was not very senior in the organizational hierarchy – he was a Director while other function leaders were VPs or SVPs. He was not part of John’s weekly leadership meetings and was invited only when IT issues were to be discussed. Therefore, his access to leadership was limited to John, his immediate supervisor. He and John had a scheduled monthly meeting to review the IT function.
The incident
On 28th Oct in the evening, at about 8 PM EST, Greg received a call from Swami, his cloud lead. Swami was alarmed as he had just realized that he could not perform any activities in the Artys cloud account. None of the other team members could do that either. It was strange as the servers seemed to be running and the customer services were operational. The first thing he did when he noticed the issue was to call Greg and inform him.
They continued on the Teams call, and Swami shared his screen with Greg for joint discovery and troubleshooting. They confirmed that the SaaS services were indeed running normally. Then, Swami noticed a mail from the cloud provider that had been sent a few hours earlier when it was close to midnight in India. The gist of the email was that the cloud monitoring systems had observed an abnormally sharp spike in resource utilization, leading to significant cost escalation. Automatic protection mechanisms on the cloud were activated, shutting down the offending servers and locking the account. The remaining running workloads were not affected, but no changes could be made to the environment.
Their first priority was to regain access to the cloud environment. As it was suspected to be a cyberattack, there was a lengthy process of internal approvals to be followed within the cloud provider, which took almost 3 hours before they unlocked the account.
Swami checked, and more than five thousand new virtual machines were created overnight. After conferring with Greg, he deleted all of them. It was late night for Greg, and he advised Swami to check if this was a human error or a malicious activity before he retired for the day.
Swami found that additional resources were created by a privileged service account that had no other activity in the last 30 days. The account was created earlier for an enterprise system that was no longer in use. This did not appear to be a human error. Checking network logs, huge outbound traffic was logged to certain IPs which were identified as known targets for crypto mining.
He immediately disabled the account and did not find anything else out of the order.
He and Greg conferred in the EST morning, and both agreed that the network traffic strongly pointed to a crypto mining attack, as no other traces of any unwanted activity were seen.
Swami – “As there is no impact to the environment, shall we consider this issue resolved?”
Greg thought for some time and said, “We may be right about crypto mining, but we must investigate deeper to rule out other possibilities. Not sure we can do it ourselves. Let me inform John and get Skylabs engaged.” Skylabs was their cyber forensic partner.
Greg updated John on the development, who was not unduly worried as there was no business impact. He asked if it was really necessary to engage Skylabs. From a past engagement, he remembered that they were expensive, and he had to approve an invoice of sixty thousand dollars. Greg was insistent it was needed and John was fine with his judgement. His only suggestion was to inform the rest of the leadership team for awareness.
Greg sent a summary email to the other leaders. Josh called him back immediately after reading it and said: “Greg, this could be serious stuff. Keep me updated on the investigation. Let’s meet as soon as you have the findings.”
Skylabs thoroughly investigated the available evidence. Unfortunately, the servers created by the intruders could not be analysed as they had been deleted by IT the same day during the clean-up. Skylabs found traces of intrusion in one of the firewall logs dated 43 days earlier, but could not correlate those with the compromised account as account access logs were retained only for the last 30 days.
Skylabs finally reported that there were no visible signatures of data exfiltration in the available evidence. However, it could not conclusively establish that no data were exfiltrated because of the lack of logs. It highlighted the unavailability of older access logs and machines created by intruders in its interim report.
Greg informed Josh and sent him a copy of the report. Josh shared it with Mahesh and told him this was serious and customers had to be immediately informed. Mahesh thought the lawyer was taking an impractical stand and blowing it out of all proportion. After a heated argument, which was inconclusive, Mahesh suggested that this be presented, discussed, and decided in a leadership meeting. He set it up and insisted that John join the meeting.
The meeting
Greg had provided beforehand an executive summary along with the formal report from Skylabs. He was apprehensive that this meeting was to review the incident and that IT might get the blame.
John finished a call on his mobile, picked up his coffee, and began, “Mahesh why am I needed in this meeting? I thought we were lucky to have escaped with no harm. I am concerned the breach happened, but we can take it up with Greg later. I am terribly busy preparing for a board meeting this week.”
Mahesh leaned forward and said, “John this meeting needs your presence as Josh wants us to inform all SaaS customers of the breach.” He paused and added, “I think it will be a disaster.”
John puts his coffee mug down and stared, “All customers? Yeah it will be a disaster. But why Josh? Help us understand why.”
Josh removed his glasses and said, “Look, I am not technical so let me first summarize what I understand. Our network was compromised and the intruders potentially had access to our customer data. We haven’t conclusively established no data was accessed. Greg, is that correct?”
Greg nodded and replied, “Yes Josh, but there is a fine line here. We haven’t found any evidence of access either.”
Josh: “But you can’t prove there wasn’t any!”
John asked, “So? Still not clear to me Josh.”
Josh explained, “We are obligated to inform our customers in case of a data breach, and we may already be late.”
Mahesh joined the discussion, “But, that’s the point John! We do not have a data breach! Why shoot ourselves in the foot and invite trouble by opening a Pandora’s box!”
Shankar added with a serious look, “Let’s also think of the huge amount of work it will generate for us. Every customer CISO will want details of how we were compromised. Our internal weaknesses will get exposed.”
Mahesh added, “John, remember the case last year when a customer terminated our contract over a failed penetration test?”
Josh said, “Team, I understand it won’t be comfortable, but we have to do what we have to do.”
John said with a puzzled look, “Guys, I am still not clear. Let’s break this down. Josh, tell us what is the exact wording in the contracts.”
Josh was annoyed executives didn’t know their contracts and came unprepared for such an important discussion. He explained in a patient tone, “Firstly our contracts are not the same for all the customers. We insert custom clauses as needed …”
John interrupted, “Ok, so which one were you talking about?”
Josh said, “I was coming to that. We have standard contracts for most of the customers. Let me read it out. You can also see it on my shared screen – “ … in the event of a security breach of Artys systems, that results in or threatens to result in unauthorized access to <customer>’s data, Artys will inform <customer> of such a breach within ninety six hours of Artys being aware of the event …. ””
Mahesh rolled his eyes and said, “This looks pretty ambiguous, typical obtuse legal language!”
Josh said with a scowl, “My friend, this obtuse language is what keeps the business running with your customers!”
John clapped his hands and peered at his screen, “Guys, come back to the point. As I read, it does appear ambiguous to me. The breach has not resulted in anything here, so this clause should not be applicable.”
Josh said slowly, “Let me try to explain the legal position again in simple terms. Let’s go over it again. We had a breach, do we all agree?”
Everyone nodded and he continued, “Let’s go to the unauthorized access part. We don’t know it happened, but we know there was a threat to the customer data as the intruder was in the network. So we meet all the conditions for disclosure.”
There was silence for some time as everyone read the clause again and absorbed it all. Finally Shankar, who generally avoided arguments and was known for his incisive and logical approach, spoke up, “I am no lawyer but I think this is vague and broad. What is threatening is subjective, isn’t it Josh?”
Josh replied, “There was an intrusion, attackers had admin access to the network and they could do anything. If that’s not a threatening situation, I don’t know what is!”
Greg who had been silently listening so far suddenly spoke up, “I don’t know if you guys noticed, but the network is segmented into customer production, customer non-production and internal Artys. The intrusion happened in internal Artys.”
John tapped his fingers on the desk and said, “How’s that important Greg. Explain in simple terms please.”
Greg: “What I meant was the attackers didn’t access customer network so their data was not threatened.”
Mahesh visibly straightened and said, “Why were you silent all this while? Josh that changes things.”
Josh was thoughtful for a moment and said, “Maybe. Greg, does the forensic report confirm they didn’t?”
Greg said, “Let me open the forensic report and double check on this point.” He did so and made a face. “No there was no visible access, but the report calls out that it was inconclusive.”
Josh: “So they could have, if they wanted to, and we just don’t know?”
Greg said with a slow nod, “Yes.”
John looked up from his laptop screen and said, “I am looking at the report. Why can’t we prove conclusively? Greg, why don’t we have logs older than thirty days and why the machines were deleted?”
Mahesh muttered, “We wouldn’t be in this situation if IT had not messed up in the first place.”
Greg leaned forward and replied, “We had to ensure customers are not impacted, and had to focus on running operations. It is easy to say in hindsight this should have been done or that, but that time we wanted to get rid of the malicious content as soon as possible and secure the business.”
John: “And what about logs Greg?”
Greg threw his hands in the air and said, “John I am doing what I can. Extended retention costs money. I have made business case for a log archival system for last two years without success.”
John went quiet as he knew it was true.
Shankar spoke again, “There is no point blaming IT right now. Let’s come back to the breach. Josh, I think we have enough ambiguity in forensic report and contract to legally defend our position.”
Josh: “I am no technical expert. If you all believe we can explain and defend the clause around ‘threatening’, I am good. But we are still missing the point.”
John: “And what is that?”
Josh: “Gentlemen, what if any customer data was actually stolen, and pops up publicly after a few months? We aren’t really sure nothing was stolen.”
John: “What happens then?”
Josh: “We will be looking at a huge legal liability. We are probably talking tens of millions of dollars here. Many organizations of our size don’t survive this.”
No one spoke for some time as they contemplated the implications of what they had heard.
Mahesh broke the silence, “Josh how about some legal advice to manage the situation? Surely there is some technicality in the contracts we could use?”
Josh shook his head and said “Guys, my jobs as a lawyer is to give you the right advice. I can’t help it if it is not what you like to hear.”
John rubbed his forehead and said, “This is so difficult. The upcoming board meeting is to finalize the next funding round, which is needed for survival. The board won’t like this at all. By the way, I haven’t even informed them yet and have to do now asap.”
Mahesh: “But John, this is not related to funding.”
John: “Of course it is. Once the knowledge of the incident is public and if we lose even one large customer, the investors will back out, and our valuation will get destroyed.”
There were no comments, and heavy silence hung in the room. John looked around the table. The board meeting was only days away, and the clock was ticking if the customers were to be informed.
Finally he spoke, “What should we do?”
Discussion questions
Help the Artys team decide on the way forward by weighing their options. They are in a difficult situation, and both the choices seemingly have serious risks. It is recommended to familarize yourself with the concepts of security climate, agency theory and bounded rationality (Chan et al., 2005; Eisenhardt, 1989; Simon, 1955).
Put yourself in the situation of John, the CEO, and make a brief plan that will be shared with the board. The plan should address the following points and provide convincing logic for the course of action chosen. (a) Will customers be communicated with, and if so, what will the message be? (b) For the course of action chosen, a list of anticipated risks and their mitigation plan. (c) Anticipate how customers will respond, and what contractual remedies they might seek. (d) Explain why this decision is being taken in the backdrop of an approaching 96-hour deadline. (e) Acknowledge the gaps in governance and response, and what could have been handled better – both before the incident and after.
Footnotes
Ethical considerations
There are no human participants in this article and informed consent is not required.
Funding
The author received no financial support for the research, authorship, and/or publication of this article.
Declaration of conflicting interests
The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Data Availability Statement
Not relevant.