Customer security incidents can happen many different ways: from a lost or stolen device, ransomware attack or a data breach notification from a trusted third party. Frequently, the first point of contact will be the IT service provider’s or MSP’s customer facing helpdesk or onsite resource - the “something’s wrong” call can happen at any time.
For the IT service provider and MSP front-line staff the pressure to follow the right process can be considerable, especially when the customer is in near hysterics. Just like police or fire paramedics responding to a call, information needs to be rapidly acquired, documented and an assessment made as to the appropriate response. Over-reaction can cause additional stress and a misallocation of resources. Under-reaction, and the IT service provider’s or MSP’s staff may be accused of “not caring” about the customer. In most cases, the front-line staff have a short period of time to determine the right course of action.
There are proven strategies and an abundance of resources on the internet about how to manage a security incident. Unfortunately, many of them fail to address the potential complications between an IT service provider or MSP and the customer – especially in a time of crisis. What can be considered an absolute truth is a security incident at a customer’s site has the potential to damage the IT service provider or MSP's business relationship. As crass as it may sound, documentation becomes even more critical if a series of unfortunate incidents results in accusations of negligence from the customer and a potential of litigation.
- Preparation - forewarned is forearmed. This expression originated as a Latin proverb and is perhaps the most succinct way of suggesting the odds of a security incident happening at a customer’s site is a likely occurrence. Thinking about the various threats and risks and preparing for a security incident significantly reduces the chances of a poor response. Anticipating a lost or stolen device, a ransomware attack, a data breach notification or a significant Internet outage physical or DDOS will set the stage for a professional response from the front-line staff.
- Detection/Identification – not all security incidents are straightforward situations; and some situations may not even require an immediate response. When a call comes in to the help desk the handler has to ask a number of questions to determine if an actual security incident is unfolding. Some incidents are pretty blatant: Ransomware is a classic example; a lost or stolen device may turn up at some point in the near future. It’s important to gather all the information available into the service ticket. Engaging the manager or supervisor is usually advisable when the call handler is reasonably certain a security incident is occurring – declare the security incident and raise the priority of the ticket appropriately.
- Analysis/Communication – gaining a clear picture over a phone call or email, from panicky and perhaps non-technical customers of what is going on may be difficult. Consulting the remote monitoring and management system, customer documentation, network details and security tools could yield details which may be valuable. Perhaps an invoice was unpaid, and a customer was “cut off” from a vital service, or the internet facing IP address of the customer has had its reputation lowered, due to a misconfiguration. Any number of non-malicious options need to be explored, all too frequently a security incident turns out to be a human mistake.
- Containment/Fix – resolving the incident successfully should be the focus of all involved. If Ransomware has broken out, taking systems offline or segmenting the infected systems from spreading may be part of immediate actions to take. A customer concerned they clicked on a malicious link may need a deep anti-virus scan and/or network layer analysis to see if there are Indications of Compromise on the workstation. The steps taken to contain or resolve the incident should be put in the service ticket in detail - as well as clear and accurate time recording from the start to the end of the security incident.
- Lessons Learned – this is perhaps the most critical part of the security incident response. Security incidents are bad for customers and bad for IT service providers and MSPs – they can put your firm’s reputation at stake. Analysis of the security incident may identify proactive steps which can be taken to prevent future security incidents. From end user security training to new technology - such as application whitelisting in the case of ransomware attack – may reduce the potential of a future security incident from occurring - there may even be a security project opportunity for the IT services provider or MSP.
For the IT services provider or MSP the emphasis on process, documentation and timekeeping cannot be over stated. For larger MSPs there may be a number of security incidents in various stages unfolding at multiple customer sites – triaging and responding is time and labour intensive. When thinking about creating an incident response for customers, think about how to scale the response with the resources you have – you may need to partner with a firm to handle more than one security incident at a time.
Be the first to hear all of Harmony's latest news by signing up to our newsletter.
About the Author: Ian Thornton-Trump, CD, CEH, CNDA, CSA+ is an ITIL certified IT professional with 20 years of experience in IT security and information technology. From 1989 to 1992, Ian served with the Canadian Forces (CF), Military Intelligence Branch; in 2002, he joined the CF Military Police Reserves and retired as a Public Affairs Officer in 2013. In Canada, Octopi Managed Services Inc. delivers managed security services to high profile legal firms and in the UK, Octopi Research Labs Ltd. undertakes security consulting and threat intelligence engagements. As the Cyber Vulnerability and Threat Hunting Team Manager for Ladbrokes Coral Group plc. Ian has an in-depth understanding of the threats, small, medium and enterprise businesses face on a daily basis. Follow Ian Thornton-Trump on Twitter