Managed services providers and their customers have become increasingly popular targets for cyber attacks.
Ransomware criminals, in particular, have singled out MSPs and their customers as prime candidates for extortion, and they've adapted their tactics to create nightmare scenarios.
Rather than simply encrypt the MSP's systems, attackers use compromised MSP employee credentials to hijack the MSP's software tools and use them to deploy ransomware to all of the MSP's customers at once.
Starting in 2019, attackers began gaining access to a wide assortment of MSPs and abusing a host of popular MSP-focused software, including Kaseya VSA, Webroot, ConnectWise Control, and Continuum. That year, attackers also gained access to a small number of NinjaRMM MSP customers through methods that had nothing to do with NinjaRMM. In each of those cases, we saw no indication that NinjaRMM was breached. Instead, evidence suggested MSPs were compromised through other channels, and attackers then used the MSPs' credentials to conduct unauthorized actions with their tools.
To the credit of the MSP community, these incidents prompted a channel-wide push for stricter security protocols. Specifically, enforcing the adoption of 2-factor authentication has had a dramatic impact on reducing successful attacks (during our recent 2020 MSP Security Summit, Coveware CEO Bill Siegel said his firm has yet to respond to an incident where 2FA was enabled across the board).
But while progress has been made, there is still a very real sense of urgency. The initial success of MSP account hijacking has encouraged attackers to continue focusing heavily on compromising MSP networks. If you're an MSP you unfortunately need to be operating under the assumption that it's only a matter of time before similar attack attempts are directed your way.
What to do now (if you haven't already)
It's vital that MSPs take the earliest opportunity to enable 2FA on any software they're using, along with email. But you really shouldn't stop there. We've created the following checklist to provide MSPs with a laundry list of specific things they can do to reduce their attack surface and improve their ability to prevent, detect, and respond to attacks.
The goal isn't to overwhelm you by suggesting you need to take action on all of these recommendations at once. Instead, we simply want to give you a list you can refer back to and gradually work your way down as time and priorities allow. After all, improving security isn't a one-and-done activity, it's an ongoing process. Just remember, any improvements you can make now will be far less time-consuming/expensive than dealing with an active attack, so don't put off getting started.
Want a PDF copy of the checklist you can refer to later? Download it here.
Note: These recommendations obviously aren’t comprehensive. Depending on your specifics (size, infrastructure, etc.), some may not be appropriate for your business. Security isn’t one-size-fits-all, and what is critical for some may be overkill for others. Do what’s practical, take a layered approach, and remember, when implementing new controls it’s always a good idea to test them first to avoid unintended disruption.
What This Checklist Covers
- Restrict Access Across Your Network
- Protect Your Users and Endpoints
- Be Ready to Detect and Respond to Security Incidents Quickly
Restrict Access Across Your Network
"The earth isn’t flat and your network shouldn’t be either."
— Catherine Pitt, VP Information Security Officer at Pearson
Many of today’s attacks are designed to land and expand throughout victims’ networks. To prevent that you need to establish barriers between your users and assets.
- Actively inventory all network assets and classify them by risk
Here's a walkthrough to get you started.
- Use strong, unique passwords
They should be case-sensitive and made up of letters, numbers, and symbols. They shouldn’t be shared or reused. From a practical perspective, that means the right password management tool is a must (no, a spreadsheet doesn’t count). There are plenty of options that allow you to securely store passwords, manage permissions, audit use, and monitor sessions. They also solve the problem of what to do when you have a tech you need to fire or who suddenly leaves.
- That includes using unique local admin passwords
Microsoft’s Local Administrator Password Solution (LAPS) can help make this manageable.
- Don't save credentials in browsers
It's incredibly common for attackers/malware to scrape browser caches.
- ENABLE MULTI-FACTOR AUTHENTICATION WHENEVER POSSIBLE
This is particularly key for preventing the current slew of ransomware attacks hijacking MSP software tools. There are a wide variety of 2FA/MFA tools out there to consider.
- Refrain from using default usernames
No “admin,” “administrator,” “default,” “root,” “user,” etc.
- Eliminate unnecessary use of elevated privileges
Live by the principle of least privilege. Ex: Techs should use standard, non-admin account by default, and a separate admin account only as necessary, ideally from a privileged access workstation.
- Create buffers between different levels of privileged access
Microsoft advises adopting a tiered model composed of three levels of admin accounts that each control a different category of assets (domains, servers, and workstations).
- Apply “least privilege” to service accounts
Creating service accounts for specific applications can help you isolate damage should a single account be compromised, but only as long as you avoid common mistakes. Disable services that you don’t require, and consider using Group Managed Service accounts (gMSA) to make managing service accounts easier and more secure.
- Remove end users from local admin group
Another one of the most basic security controls that gets routinely overlooked or undermined in real life. Here’s a simple walkthrough of how to remove local administrators using a GPO, along with advice on how to address any misconceptions or pushback you may run into from execs and others.
- Audit systems for inactive user accounts
It’s estimated that one third of user accounts are inactive but still enabled. All those untended access points represent a big security risk. Audit regularly, and have a clear policy in place for disabling and deleting accounts when users leave the company.
- Block lateral movement between workstations
A growing number of today’s attacks don’t just infect single workstations. They’re designed to land and expand. By using Active Directory, Group Policy, and Windows Firewall you can prevent workstation-to-workstation communication while still allowing access from your privilege access workstation.
Lock Down Your Remote Management Tools
Not only are remote access capabilities critical to your business, there are also few things an attacker would love to hijack more.
- Restrict access to remote management tools
Limit their availability strictly to the people who can’t do their jobs without them.
- Use strong, unique passwords AND multi-factor authentication
- Limit what remote accounts have access to
Always be thinking least privilege, especially when working with clients in regulated verticals or that deal with PII or other sensitive information. You may find yourself needing to prove that your techs never had the ability to access that information.
- Don’t log into workstations with domain administrator accounts
Doing so risks attackers harvesting the DA credentials should any of those workstations be compromised. Domain admin accounts should few and far between, and exclusively used to log into domain controllers (no workstations).
- Keep remote management software up-to-date
Regularly apply updates and keep special lookout for any patches addressing vulnerabilities that could provide attackers with remote code execution or unauthorized access.
- Enable centralized logging/monitoring and alerting for remote access sessions
Capturing information on remote access sessions and activities will allow you to conduct audits, spot anomalies, and investigate and respond to any suspicious activity.
Secure Remote Desktop (RDP)
Securing RDP may be basic security 101, but failure to do so continues to be one of the leading causes of compromise. A quick Shodan scan shows millions of systems currently exposing RDP. They’re undoubtedly being subjected to brute-force attacks. Once cracked, access to compromised accounts can be purchased for a handful of dollars on dark web marketplaces.
Compromise via RDP has been the go-to attack vector for numerous ransomware variants, including CrySiS/Dharma, Shade, and SamSam, the ransomware used to infect Allscripts, numerous hospitals, and the city of Atlanta.
- Don’t expose RDP (or any internal resources) to the Internet unless absolutely necessary
Even then, question whether there’s a better way to do what you need.
- Use port scanners to identify RDP (and other ports and services) exposed to the Internet
Use scanning tools like Nmap, masscan, or Shodan. Rest assured the bad guys are already doing this, so take a few minutes to see your network through their eyes. Another tool that walks you through basic port scanning is ShieldsUP.
- Identify systems that have been compromised with RDP backdoors
One of the most common ways of achieving a backdoor is by abusing the Windows Sticky Keys feature. Two different scanning tools for identifying backdoored RDP servers are available here or here.
- Disable RDP on machines that don’t need it
That reduces the risk of attackers leveraging one compromised machine to access others on your network.
- Remove local admin account access to RDP
All admins are able to log in to Remote Desktop by default. This post from Malwarebytes walks through how to remove local admin account access and create a restricted user group in the Group Policy Management Console, instead.
- Use strong, unique passwords and multi-factor authentication
MFA is a good idea regardless, but if you absolutely have to have RDP exposed, it’s a must.
- Implement an account lockout policy
The number of failed attempts required to trigger a lockout is up to you, but as a general baseline, Microsoft recommends 15-minute lockouts after 10 bad attempts.
- Log off disconnected and idle sessions
This may not be popular, but it’s an important mitigation to prevent sessions from being hijacked.
- Restrict RDP access using firewalls, RD Gateways, and/or VPNs
By using a firewall you can restrict RDP access to whitelisted IP addresses. RD Gateways are more comprehensive. They allow you to restrict not only who has access, but what they have access to, without needing to configure VPN connections. More on RD Gateways here.
- Leave Network Level Authentication (NLA) enabled
NLA provides an extra layer of authentication prior to establishing a remote connection. More on how to check your Group Policy settings to confirm NLA is enabled here.
- Change the default listening port (TCP 3389)
This won’t hide RDP from determined attackers, but it will raise the bar and help protect you from automated attacks and lazy ones. Microsoft explains how to make the change here.
- Securing Remote Desktop (RDP) for System Administrators
- FBI / DHS Alert: Cyber Actors Increasingly Exploit the Remote Desktop Protocol to Conduct Malicious Activity
- Organizations Leave Backdoors Open to Cheap Remote Desktop Protocol Attacks
Protect Your Users and Endpoints
"Show me a malicious email and I'll show you someone who will click."
— Ancient infosec proverb
The vast majority of attacks target the most vulnerable part of your network: your users. Here are best practices for securing their devices and protecting users from themselves.
- Use endpoint security software that utilizes machine-learning and/or behavioral analysis
These days, few AVs rely solely on signature matching. Instead, the majority of AV vendors have either incorporated machine learning detection algorithms into their general offering or as an additional (pricier) product line (see our guide on EDR and NGAV tools here). These solutions are markedly better at blocking malware that’s new or polymorphic, but the downside is they can generate a considerable number of false positives. In addition, they’re often still blind to attacks that abuse legitimate system tools or utilize other “fileless” techniques — another reason security is all about layers.
- Keep endpoint systems and software up-to-date
Often easier said than done considering there were more than 15,500 CVEs published in 2018. Windows updates alone can be a beast, nevermind third-party applications. Make sure you’re automating patch management as much as possible through your RMM, and that you actually trust it to apply patches successfully. Conduct regular patch audits to identify machines that may be vulnerable. 57% of data breaches are attributed to poor patch management.
- Develop a standard operating procedure for auditing your firewall policies
Make sure you’re protecting your perimeter by maximizing your firewall’s inspection and filtering capabilities.
- Utilize DNS filtering to protect against known malicious websites
DNS filtering solutions can provide protection for users even when they’re off the network.
- Strengthen your email security
With a staggering 92% of malware delivered via email, having a good spam filter is obviously a must. Unfortunately, it’s not just malware you need to worry about. To help prevent phishing and business email compromise (BEC) attacks it’s a good idea to set up DMARC, SPF, and DKIM to protect your domain from being spoofed. Here’s a setup walkthrough and a free DMARC monitoring and reporting tool that can help.
- Provide security awareness training to teach employees how to spot malicious emails and websites
Users will be users. They’re going to click things they shouldn’t, but if you’re not training them it’s hard to blame them. Especially as malicious emails continue to become increasingly convincing. Start by educating them on the classic warning signs, showing them real examples, and sharing basic best practices like hovering over links. Then consider moving on to phishing simulations and more formalized training
- Utilize a reliable backup solution and actually test recovering from backups at scale
Having multiple restore points and offsite replication is key, as is doing regular tests to ensure your backups are configured and working properly. Remember Shrodinger’s Backup: “The condition of any backup is unknown until a restore is attempted.”
Windows system hardening
Many of today’s attacks attempt to abuse built-in tools and functionality. This tactic of “living off the land” helps them bypass defenses and evade detection by blending in with legitimate admin activity. Here are steps you can take to mitigate:
- Guard against credential dumping
For Windows 10 and Server 2016 machines, consider enabling Credential Guard. You can also limit or disable the number of previous logon credentials Windows will cache (the default is 10). Here are instructions for disabling credential caching on older systems.
- Disable or restrict PowerShell
Attackers abuse PowerShell for a wide variety of tasks, from downloading and executing malware to establishing persistence, achieving lateral movement, and more (all while avoiding AV detection). It also comes enabled by default. If there’s no need for PowerShell to be on a user’s machine, get rid of it. If that’s not an option make sure it’s the latest version, disable the PowerShell v2 engine, and use a combination of AppLocker and Constrained Language Mode to reduce its capabilities (here’s how).
- Restrict the launch of script files
PowerShell isn’t the only scripting language and framework attackers love to abuse. Prior to Windows 10, Microsoft recommended making changes to the registry so a warning prompt was issued before allowing .VBS, .JS, .WSF, and other script files to run. Windows 10 systems can utilize AppLocker to block script files with more granular control.
- Use AppLocker to restrict applications
Whitelisting isn’t for everyone — it can be challenging for some teams to manage and maintain — but in environments where it’s feasible it can be a very effective layer of security by limiting what applications can run under what conditions. You can find tips for getting started with it here.
- Block or restrict “Living-off-the-Land” binaries (LOLbins)
To bypass AV and whitelisting solutions like AppLocker, attackers are increasingly abusing native Windows tools. Built-in programs like certutil, mshta, and regsvr32 should be blocked or prevented from making outbound requests using Windows Firewall rules. The same goes for legitimate data-transfering tools bitsadmin and curl. Find a more comprehensive list of “LOLbins” here.
- Utilize Windows Firewall to isolate endpoints
In addition to blocking the LOLbins listed above from making outbound requests, you can use the Windows Firewall to cut off some of the most commonly abused paths for malicious remote access and lateral movement (such as gaining access to file shares via SMB). Here’s a great walkthrough.
- Restrict or monitor Windows Management Instrumentation (WMI)
WMI ranks up there with PowerShell in terms of its utility, which means you should be monitoring for potential abuse of it, too. For any cases where remote WMI isn’t necessary, consider setting a fixed port for it and blocking it.
- Use highest user account control (UAC) enforcement levels whenever feasible
UAC can place a significant roadblock in the path of attacks attempting to elevate privileges. Consider adjusting Windows 10 policy settings to automatically deny elevation attempts for standard users and prompting for consent on the secure desktop for admins (guide to UAC group policy settings for previous Windows versions available here). In addition, strongly consider enabling Admin Approval Mode for the built-in admin account. In addition to mitigating privilege escalation attempts, that will also cause any attempts to abuse PsExec (legitimate admin tool in Microsoft’s Sysinternals suite) to fail.
- The Increased Use of PowerShell in Attacks
- Endpoint Isolation with the Windows Firewall
- Living Off the Land Binaries and Scripts (and also Libraries)
Securing Microsoft Office
Malicious Office documents continue to be one of the most popular and successful delivery vehicles for malware. The key to mitigating that threat is to disable or restrict the following features.
- Disable or restrict macros
Hiding malicious macros inside Office documents is one of the oldest tricks in the modern attacker’s playbook, and continues to be popular and successful. If macros aren’t utilized in your organization consider using Group Policy settings to disable them without notification and disable VBA for Office applications altogether. If you do need to run macros under certain conditions, restrict them by only allowing signed macros and blocking macros in Office documents downloaded from the Internet — walkthrough (Office 2016); Group Policy Administrative Template files (ADMX/ADML)
- Disable or restrict Object Linking and Embedding (OLE)
Walkthrough for blocking activation of OLE packages via registry changes; walkthrough for blocking activation of OLE / COM components in Office 365 via registry change; walkthrough for disabling data connections and automatic update of Workbook Links via the Trust Center.
- Disable Dynamic Data Exchange (DDE)
Walkthrough for disabling Dynamic Data Exchange Server Lookup / Launch via registry changes; walkthrough for disabling via the Trust Center
Be Ready to Detect and Respond to Security Incidents Quickly
"The downtime and disruption caused by the average ransomware incident lasts for 7.3 days."
It’s not enough to work on preventing attacks. You also need to have the right capabilities and policies in place to identify, contain, investigate, and remediate compromises quickly.
Note: There are basic things you can do here, but on the advanced end, it often involves utilizing complex tools, combing through logs, and providing 24/7 monitoring/response capabilities. Depending on your expertise, bandwidth, and requirements, you may need to consider outsourcing.
- Establish a network performance baseline so you can identify anomalies
- Use your RMM and/or a SIEM to configure centralized, real-time network and endpoint monitoring
- Take advantage of out-of-box alert configurations and create templates for standard use cases (workstations, servers, etc.)
- Develop standard operating procedures for addressing most critical and most common alerts
- Reduce noise by eliminating alerts that lack severity and aren’t actionable
- Consider monitoring key Windows Event IDs — start with the lists here and here
- Consider utilizing an endpoint detection and response (EDR) solution
- Enable and configure the right system logs to assist in your own or outsourced digital forensics and incident response (DFIR) — see these cheat sheets for Windows
- Store logs in a central, isolated location
- Determine if you need to outsource management of all or some of the above to a managed detection and response (MDR) provider
Create an incident response plan
When a security incident does occur you need to be able to act quickly under pressure. That takes clear guidelines and effective planning.
- Define what constitutes a security incident
- Establish roles, responsibilities, and procedures for responding to incidents, including disaster recovery
- Identify escalation options should incident require more extensive/expert response and recovery than you can provide
- Have a plan for communicating internally, with customers, authorities, and the public (if necessary) — better yet, have templates at the ready
- Understand compliance requirements regarding incident disclosure and reporting — HIPAA Breach Notification Rule; GDPR data breach notifications FAQ
- Run fire drills
- Additional resource: Incident Handling: First Steps, Preparation Plans, and Process Models from ERNW
Closing: You Don't Need to Boil the Ocean in a Day
Depending on how much you've already invested in security, this list may feel overwhelming. If that's the case just remember, security isn't something anyone gets 100% on. Things are always changing and the goal isn't to become magically bullet-proof, it's simply to make sure you're consistently taking small steps forward.
Focus on doing a few things from this list at a time. Or even just one thing. Then do another. Aim for incremental progress. Everything you do can have an impact. If you're lowering your risk or raising the bar for attackers, even slightly, then you're doing your job.
Want a PDF copy of the checklist emailed to you? Download it here.