Review: April 2021 NSA Cybersecurity Advisory

Did we just go back into pre-Stuxnet times?

It sounds ominous to try and “stop malicious cyber activity against connected operational technology”, but is the messaging correct?   It’s certainly well-meaning and aligned with the ideals of OT/ICS, but at this point – private industry (which represents most of the US critical and manufacturing sectors) has been moving for nearly FORTY years towards connectivity and data in order to complete its mission.  It’s a bit of a too little too late notice, and the advice has consequences.

So let’s break apart the executive summary first:

A significant shift in how operational technologies (OT) are viewed, evaluated, and secured within the U.S. is needed to prevent malicious cyber actors (MCA) from executing successful, and potentially damaging, cyber effects. As OT components continue being connected to information technology (IT), IT exploitation increasingly can serve as a pivot to OT destructive effects. Recent adversarial exploitation of IT management software and its supply chain has resulted in publicly documented impacts across the U.S. Government (USG) and the Defense Industrial Base (DIB). Malicious cyber activities directed at OT also continue to threaten these networks.

Well, this reads fairly well, and is an accurate account of where we are today.

The majority of threats are delivered through poorly separated and protected IT networks, and the bad guys wander in likely through that, or IT systems providing connectivity to the OT segments.  However, the specific mention of saying recent adversarial exploitation of IT management software and supply chain should be broadened – to include OT management software and supply chain.  Tons of kits now use open-source components, embedded operating systems etc. – and as we know, it’s adulterated, abandoned, and left insecure for periods of at least 5-10 years.  IT-only isn’t inclusive enough if you want to protect the grid from being backdoored — or from product developers using the same code within the IT supply chain.

The second paragraph of the executive summary:

This paradigm shift applies to the stagnant OT assets and control systems installed and used throughout the USG and DIB, many of which are past end-of-life and operated without sufficient resources. To evaluate and improve the cybersecurity of connected OT and control systems, NSA recommends that National Security System (NSS), Department of Defense (DoD), and DIB network owners perform a detailed risk analysis prior to creating cross-domain connections (e.g., IT-to-OT, Internet-to-OT) and for all currently connected OT.

Seems likely again, and one would have thought this was mandated or standard practice, and EoL assets in OT should not be a new concern for any of those branches of government/military.  And the third paragraph?

Following the steps below will enable OT owners and administrators to evaluate risks against their systems and use that knowledge to guide network changes with current resources to realistically monitor and detect malicious activity. Without direct action to harden OT networks and control systems against vulnerabilities introduced through IT and business network intrusions, OT system owners and operators will remain at indefensible levels of risk.

Risk management in the private sector is tricky.  To date – there has been little incentive to really remediate the risks, downtime is expensive, commitment has been rare, but also due to the fact that asset owners have had to absorb the risk passed onto them by the OEMs and vendors (also mostly US-owned) who created the products in the first place; I admit revenue vs. time vs. effort is a fundamental feature of capitalism, but it has gone unchecked in ICS/OT.

And while it is true that asset owners should have the resources, technology for anomaly detection – most private industry is resource-strapped, and needs IT-esque systems such as ERP, billing, and connected technology because they aim to be profitable and competitive.  Isolation is the last resort for most organizations today after they have spent copious amounts of money trying to modernize or go digital.

Alright so let’s look at the steps and see what was good about the NSA Cyber Security Advisory:

• Risk management and evaluation via multiple criteria other than revenue was a step in the right direction. It is far too common (almost weekly) that we find small-to-large customers alike asking about which assets or facilities they should prioritize.  Mandating this is crucial and likely expected.
• Recommendations for more golden images – I can’t express how this is a foundational activity moving forward and aligns well with the suggestions of baselines.
• Monitoring/removing remote access, however, this assumes you know about it and unauthorized solutions that can call home such as TeamViewer are not installed – so this one is more so/so
• Risks need to have measured responses – assess, prioritize, and assume an ever-evolving improvement plan. Fair enough
• Limit access between systems, network segments, applications etc. Monitoring any authentication attempts good or bad, and consider physical access. Good, but connection-focused – what about what’s internal?
• Fire drills and practice for recovery and re-installation – this should be in everyone’s back pocket. Unfortunately, it missed out on the part where you need to have configurations and sometimes spare kit.
• Mentioning that a number of tools from IT will work in OT – that’s true, but they can’t be used out of the box unless customized by experienced OT professionals and integrators.

What was bad about the NSA Cyber Security Advisory:

• The general suggestion that islands or airgaps are good is actually a huge step backward in most cases. They are acceptable under certain conditions, and not a cop-out to properly own and maintain a system.  In some rare cases, airgaps are perfectly OK when they are CLOSE to the affected system – asset owners will read this as oh…. I’ll just airgap the edge of the facility and be done with it.  Humans will get creative, PDFs/installers tampered – and your resources will bring it in via transient media… sounds like Stuxnet NG to me…
• Ignored the realities and focused only on networking ACLs, limiting connectivity etc… Unfortunately, tons of OT systems are within IT data centers or networks, multiple flows go between them etc… arguably they should not be going to PLCs, but they often go to at least the SCADA level.
• The focus on creating baselines just using networks and sensors. This assumes everyone has managed L3 switches (mostly) capable of span ports, a method to forward traffic, or a budget to install taps in addition to a sensor solution.  Sensors are not necessarily required if you have Netflow, MAC filtering, and syslog monitoring…
• The advisory ignored the fact that most asset owners operate in a distributed environment with large geographical distances and one of the reasons critical infrastructure has largely remained functioning is due to the past investments in remote access technology.
• Disabling remote access is more than shutting down the authorized solution, it also includes possessing detailed endpoint information which is able to list all installed software in order to detect risky remote access software.
• Suggestions about open-source tools – fine by me, but they are pretty terrible in general as evidenced during the 2^nd ICS Detection Challenge, and let’s hope the end users didn’t assume this meant they could use NMAP 😉

And what might have been missing in the NSA Cyber Security Advisory:

• IoT/IIoT – these devices are frequently deployed with direct access to the Internet/although often out of band (OOB) and stand-alone wireless.
• OT/ICS also includes building automation and safety systems. Specific language should have been noted for this case.
• Wireless systems were not noted at all in this report – countless ICS/OT systems communicate over RF, LTE, and other protocols that are non-ethernet. Good luck baselining that.
• Endpoint security as a whole – AV, whitelisting, user/role controls, patching, change control, host monitoring etc…
• Security framework suggestions were missing from the article as a whole. I am surprised that ISA62443 was not mentioned as this was an IT VS. OT-themed article.
• Transient asset management and the security required to manage this concentrated threat vector.

To end this review – the last paragraph was likely the most representative of sound thinking and should have been moved to the initial executive summary.  Prioritizing risk, by mindful of remote access risks, IT/OT interconnectivity etc.

The key to an efficient and effective OT security program is to take a comprehensive risk view of the assets and the process, something we call 360-degree risk management. 360-degree risk management means the ability to aggregate the risk information about an asset and its network and process. Then to prioritize which risks to which assets are most critical. Companies can waste significant resources trying to tackle every CVE, especially when many if the OT systems cannot easily be updated given the legacy environments. It is critical that OT security looks at things such as true network device configuration,  not just what’s on paper in the design, patch status, A/V status, backup status, risky accounts and password settings, etc. To really secure OT, organizations need to apply core system management functions.