Achieve Vendor Agnostic OT Endpoint Security

Endpoint security is the process of identifying, detecting, protecting, and responding to cyber security threats at the device level. Gartner has defined an endpoint protection platform as a solution to “prevent file-based malware attacks, detect malicious activity, and provide the investigation and remediation capabilities needed to respond to dynamic cyber security incidents and alerts.”

What are the key elements of successful endpoint security?

However, endpoint security goes beyond this definition to include the identification of the endpoint itself, which is not always rudimentary (especially in operating technology or cyber physical systems). Endpoint security also needs to have the protection of that endpoint from known vulnerabilities, which includes updating patches, hardening configurations, etc. Further, it also has the ability to prevent inappropriate access to that endpoint, its data, or its functionality through control of users and accounts.

Endpoint security is much more than anti-virus (whether past-gen or next-gen) and EDR (endpoint detection and response). If an organization misconfigures or fails to patch an endpoint system, an attacker may not need to use file-based malware to compromise the endpoint. Organizations should see true endpoint security as a comprehensive set of defensive measures.

What are the components of effective endpoint security?

There are eight components of effective endpoint security:

1. Visibility and identification. Often this fundamental element of effective cyber  security is forgotten by organizations jumping into an endpoint security project. In today’s world of remote work, growing IoT connections, and the increasing connectivity of operational technology or industrial control systems, you cannot take for granted that you have visibility into all of the endpoints in the environment. Further, you need to identify that endpoint as to what it is and its function to enable proper security.
2. 360-degree continuous risk assessment. An endpoint’s risk includes various factors: known software or hardware vulnerabilities, insecure configuration settings such as open ports and running services, insecure user and account management, etc. Endpoint security requires that these risk views be known and assessed against known threats and vulnerabilities. Without a 360-degree view, it becomes impossible to understand how to prioritize remediation actions based on the asset’s contextual information such as its role and criticality to operations..
3. Proactive remediation of known risks. Larger organizations often forget this step when considering endpoint protection due to organizational silos. Patching, configuration management, user & account management, software management, etc., are usually managed by separate teams than the endpoint security team. However, they are critical elements in the overall endpoint security mission. A great EDR solution does not make up for unpatched systems.
4. Detection of signature and non-signature-based malware and active threats. This includes applying machine learning and AI to behavioral data of endpoint information. Sometimes referred to as EDR or its latest incarnation XDR (Extended Detection and Response), this function gathers file and process data from endpoints to compare to possible malicious behavior. Robust programs integrate data from various sources (endpoint, network, user, etc.) to capture the broadest view of potential attacks and behaviors as possible.
5. Prevention of malware attacks. Traditionally, an organization achieved this through signature-based anti-virus and its relative, application whitelisting. In today’s defensive arsenal, this also includes “next gen AV” which includes prevention based on ML (machine-learning)/AI-based analysis of behaviors. However, application whitelisting still has advantages in many environments.
6. Managed threat hunting. This adds a human element to the process, whereby a team actively looks for threats based on initial indicators using a wide range of experiences and analytics based on prior behavior patterns.
7. Endpoint response. A robust endpoint security program must also define response plans should the organization discover a potential attack. Organizations must tailor these response plans to the specific environment within which the endpoint operates. For instance, an OT device will have significantly different response parameters than an employee’s office laptop.
8. Enterprise reporting and management console. In today’s distributed endpoint environment, robust endpoint security requires comprehensive visibility of all security statuses and ongoing threats and responses across all asset types at all sites. One problem discussed above is the enterprise does not get a 360-degree view of its environment. This is critical to keep up and not become burdened with administration.

What is an endpoint?

Endpoints can include a wide range of devices. The basic definition includes any device in a computing network that can process, store, or transfer data. This includes:

• Traditional endpoints:
  • Servers
  • Laptops
  • Desktops
  • Printers
• Network endpoints
  • Routers
  • Switches
  • Firewalls
  • Modems
• IoT endpoints
  • Cameras
  • Alexas/smart devices
  • Sensors/watches
• Industrial/Operational/OT endpoints
  • PLCs
  • Controllers
  • RTUs/RTACs
  • Communication equipment
  • Building controls

As evidenced by the above list, the traditional endpoint definition requires expansion and consideration of a wide range of devices that were traditionally not in the realm of “endpoint security.” Traditional anti-virus focused on devices with traditional operating systems such as Windows or Mac, but today’s endpoint environment is the “wild west,” requiring a rethink of the conventional definitions of “endpoint” to ensure comprehensive security.

Why does endpoint security matter?

Endpoint security is a critical element in “defense in depth,” a comprehensive set of security controls and approaches designed to provide layers of protection to IT and OT systems. Every endpoint is a possible attack vector into the organization. Some believe (particularly in operational technology and industrial environments) that network protection – either through “air gaps” or robust network protection approaches such as tightened firewalls or even data diodes – eliminates the need for endpoint security. Still, others argue in many of these same environments that endpoint security is not feasible, so network traffic analysis and network intrusion detection is satisfactory.

The reality is that endpoint security – all the elements mentioned above – is one of the most critical forms of organizational cyber security. Just take one example: ransomware. It targets these same endpoints. Perimeter network protections can provide some level of defense to ransomware, but when active credentials are stolen, attacks move through third-party endpoints or USB sticks that avoid firewall protections, etc. Ransomware can spread rapidly in unprotected endpoint environments, even with robust network defenses. And once a threat gets through that boundary, it can spread quickly if endpoints are not secured.

Operational technology (OT) endpoint security: The challenges

OT endpoint management is necessary to protect the world’s critical infrastructure from cyber-related threats. But in many cases, it is not deployed due to several key challenges. The unique characteristics of these networks, combined with the processes they control, make running traditional endpoint protection solutions very difficult, if not impossible. For example:

• Visibility and inventory are fundamental challenges in industrial environments, primarily due to the large percentage of legacy devices.
• Many devices in an OT network do not run Windows/Unix/Linux, but instead operate on embedded firmware without the ability to deploy traditional IT agents.
• Scanning these systems with traditional vulnerability scanners can cause device malfunction and operational outages, making timely vulnerability management rather difficult.
• Patching is both a risky and time-consuming process. Patching or updating firmware on embedded systems may require expensive upgrades of the entire control system. Even traditional OS patches are challenged by the limited ability to reboot operating systems.
• OEM vendors test and approve specific versions of anti-virus on their control systems to ensure the scanning does not interfere with their control processes, making an enterprise standard solution challenging. Updating the signatures, too, is challenging due to the need to test and confirm operational resilience with the new signatures.
• Most OT or industrial systems do not directly connect to the cloud for security and operational reasons. Therefore, many or most of the next-gen AV tools cannot operate effectively to provide the benefits of cloud-based AI.
• Automatic response and prevention, with its possible false positives, can have a significant negative impact on operational systems. Shutting down a process incorrectly on a turbine could cause catastrophic harm. Therefore, the prevention components of modern EDR and next-gen platforms often are restricted in OT.
• These endpoints operate in complex and legacy networks, making enterprise visibility and management exceptionally challenging.

As a result of these challenges, organizations often see OT endpoint security as too time-consuming or impossible.

OT Endpoint Security: The Solution

There is a way to approach endpoint security in OT that addresses these challenges and provides robust protection.

Five steps for successful OT endpoint security management:

1. Create “360-degree” risk scores and profiles for each asset.

This process begins with technology that enables deep vendor-agnostic endpoint visibility. The good news is there is a way to capture a comprehensive inventory of OT devices. Using OT-sensitive agents and agentless connections, an organization generates accurate and real-time inventories of the endpoints they need to secure.

This visibility must include 100% software inventories, full patch status on all the application software as well as OS, detailed and regular information on configuration settings, password and user/accounts, defensive tool status such as AV and whitelisting, network configuration rules and settings to understand network defenses, and asset criticality based on process and network.

This “360-degree” view of risk allows the organization to define the most effective and efficient means of remediating risks and securing a given endpoint. For instance, we obviously cannot deploy AV on a PLC, but that doesn’t mean there aren’t means to protect that asset through upstream compensating controls such as:

• locking down its workstation
• establishing a firewall in front of that device
• hardening the configuration of that device to stop the spread of a potential threat

Similarly, we may find two equally vulnerable assets, but one has multiple compensating protective controls such as application whitelisting, hardened configurations, etc. This allows the operator to make trade-offs on priorities and actions. See more on our Technology Enabled Assessment Whitepaper.

2. Execute initial endpoint security remediation plans based on the feasibility of different approaches: configuration hardening, patching, network protection hardening, locking down endpoint protection elements, etc., on an asset-by-asset basis.

Too often, organizations start with a tool (EDR or Change Management or Network Anomaly Detection or Firewalls) without a robust endpoint security remediation plan. While all of these may be helpful, the remediation plan allows the organization to step through a sequenced roadmap of actions – and technologies – that drive consistent improvement in the endpoint security management of the enterprise. Success requires a strategy that prioritizes the correct type of endpoint security for each of the risks identified.

One key element is taking advantage of the uniquenesses of OT systems. For instance, application whitelisting has fallen out of favor in enterprise/IT/cloud security because it is impossible to keep up with all of the changes required from the whitelist. On the other hand, in OT, applications do not change, and in fact, we want to limit any new applications running on the system. Therefore, application whitelisting is a very cost-effective solution for OT endpoint security.

3. Implement vendor-agnostic but OT-safe endpoint security technology: “Think Global: Act Local”.

As mentioned above, OEMs often pressure customers not to install any security software on their devices that has not been approved by the vendor. This leads to patchwork solutions. By the same token, specific endpoint security components, such as active anti-virus, can create operational risk if certain processes are stopped inappropriately or the software scanning the device utilizes too much CPU.

One successful solution is to deploy an enterprise-wide, vendor-agnostic OT-specific safe agent to conduct the OT endpoint management functions without deep scanning or active process prevention. This is used to integrate the various OEM-approved anti-virus solutions so the enterprise has a single management console, even if the vendors each select different AV vendors.

We call this approach “Think Global: Act Local.” By creating a centralized view of endpoint security, operators centralize endpoint detections, alerts, risks, etc., to a central team for analysis, response planning, etc., but – with technology – enable the OT operator that understands his or her system best to be involved in approving and perhaps testing any security response. We realize to someone in IT that this may sound crazy – this extra step of having a “man in the middle” of the response action could slow the response. Yes, it can. But it avoids the “Type II” error of stopping critical processes that may affect the safety of the overall system.

Insurers, regulators, directors and others are beginning to require clear demonstration of security improvement. Industrial operators need to show how they moved from “red” to “green” in cyber security progress, how updated their patch or backup or AV status is, whether they have dormant accounts that create risk, etc. This type of centralized, vendor-agnostic system allows for tracking, reporting and auditing on an ongoing basis.

4. Adapt XDR for OT to allow IT security teams to achieve effective and efficient multi-telemetry detection and response in OT environments.

“XDR” is often thought of as pertaining to cloud or hybrid environments. Successful organizations consider this same concept for OT as well. Because traditional EDR may not be effective on embedded devices in OT or even in purely automatic response mode on critical control systems OS-based devices, industrial security requires a wide range of telemetry and response to be effective.

The “X” may be different in OT than in the cloud. It may refer to traditional telemetry such as endpoint logs, network traffic alerts, AV alerts, etc. But in OT, it should also include device performance metrics, physical alarm data, etc. By bringing these various forms of telemetry together, the endpoint detection becomes much more robust than if we were just to monitor packets for anomalous traffic.

Similarly, the “R” or response needs to be tuned for OT. The answer to each alert cannot be to shut down the plant. More organizations need to adopt a mindset we call “Least Disruptive Response.” This is the notion that in any event, security should try to take the action which has the most negligible impact on operations. This, however, requires security has that deep endpoint visibility discussed in Point 1 and the ability to take endpoint actions in Point 2, above. This enables the security personnel to identify the threat as well as the endpoint information about that asset as well as other assets in the attack path. Then to take concrete action – at the endpoint – to stop that particular attack path. For instance, remove an account that is compromised, patch a certain vulnerability that is being exploited, remove a piece of risky software, adjust whitelisting rules, etc.

5. Establish a set of OTSM guidelines and procedures that are OT-specific but provide similar functionality as their IT brethren.

Last – but perhaps first in many ways – industrial organizations need to set their north star, their overall objective of security, as well as their expectations of maturity. This direction flows down into policies, guidelines and procedures to implement their endpoint security management. Different assets are likely to require different levels of security based on criticality, redundancy, etc.

We have seen successful clients prioritize these assets at the site level all the way down to individual assets in a plant, then design different security targets for each one.  OT Systems Management is the process of applying policies and actions to OT endpoints to ensure they are secure. This approach requires adjustments to traditional IT policies and procedures. Perhaps the most significant of these adjustments is in the area of patch management. Unlike IT policies which are to apply all security patches as soon as possible – weekly or monthly – OT policies need to address the unique operational processes that these devices manage. OT policies will need to adjust for OEM-patch approval,  matching timing of patches to outages, etc. For embedded devices, “patching” really means firmware updates which may require other control system upgrades to remain operational.

Patching is just one example. All policies will need to be adjusted to OT. Organizations will need to define the type of response time expected for the “XDR” for different types of attacks and assets. They’ll need to determine what objectives are appropriate for any standards they aim for such as CIS Top 20/18 or NIST CSF, etc. This topic is again worth its own whitepaper, but in short, establishing coordinated objectives and policies for OT endpoint security is critical.

This 5-point approach has led to significant, rapid, and demonstrable improvements in industrial organizations’ OT cyber security maturity. Further, it has avoided what we see as a coming “perfect storm” of increasing attacks, decreasing resources, and more significant reporting and auditing requirements. It’s a way to get out in front of what is coming.

How does Verve’s endpoint security management platform work?

Verve Industrial leveraged 25 years of ICS engineering experience to build the Verve Security Center (VSC) software and our Verve Industrial Protection (VIP) services to deliver a complete OT endpoint management solution to address ongoing complexities.

VSC is the only solution of its kind, built from the ground up, with ICS/OT in mind. As a team, we [Verve] have operated in plants, and deployed Emerson, ABB, Rockwell, and many other control systems. We have seen the challenges these systems present and embedded this knowledge into VSC to create a solution that is safe, effective, and efficient for OT.

The Verve Security Center includes five critical elements for successful endpoint management security:

1. A best-in-class agent-based solution tuned for the sensitivities of OT systems: lightweight footprint, no inbound ports required, no scanning of the endpoint, visibility to OT software as well as traditional IT elements. The software-based agent has been deployed for over a decade on almost every OEM brand of equipment with no operational impact.
2. A proprietary agentless device inventory, extending reach into embedded firmware assets of networking devices, relays, RTUs, IEDs, PLCs, etc. Instead of relying on network traffic to determine endpoint information, communicate directly with these devices in a safe manner to gather configurations and other asset information from these proprietary protocols so a user can see and managed 100% of their assets.
3. A low-cost, scalable architecture allows customers to “think globally, but act locally”. This architecture provides central reporting and analysis, with local control over the actions necessary to remediate the threats.
4. A unified user interface bringing together all data into a searchable and automated asset management system to provide full visibility, actionability, direct priority, asset maintenance, and sustainment planning.
5. Skilled OT resources to apply the right solution (e.g., application whitelisting) to the specific OT environments involved.

The agent and agentless service allows for a comprehensive inventory of all assets without the complexity and cost of deploying spans/taps, etc. and without the uncertainty that comes from trying to discover endpoint information from network traffic. In addition, the architecture enables OT-safe actionability to respond to risks whether those be patching, user/account management, detection and response to ongoing threats, etc. Key to this is the “Think Global:Act Local” architecture that ensures that a technician with knowledge of the process is involved before preventive actions occur. While in IT this may sound inefficient, in OT it is critical to maintaining robust process operations and safety.

Verve Security Center: The Benefits

VSC delivers true endpoint protection for all OT assets, in a safe, effective, and efficient manner. Key benefits include:

Lower total cost of ownership

Operating across vendors and integrating elements of endpoint protection into a single offering, the cost of deployment and labor to manage the protection is significantly reduced. A 100% software solution means no expensive deployment of hardware taps or additional span ports.

Deeper and broader asset visibility

VSC enables automated asset identification, inventory, and management across all OT assets. The agent gathers up to 1,000 pieces of data on every OS endpoint to provide integrated risk context, such as full patch status against OS and third-party patches, full vulnerability assessment, view of unauthorized accounts, unnecessary/risky software, configuration insecurities, etc. The agentless device inventory gathers deeper and broader asset visibility because spans/taps on network devices are not needed.

Faster time to cyber security maturity

Deployment and mean-time-to-remediation (and therefore, time to demonstrate progress on security maturity) are much faster than with passive/network-based tools. VSC’s closed-loop vulnerability management solution allows clients to immediately take actions from the platform to remediate risks and threats currently present. Unlike detection platforms, VSC offers a true management platform, incorporating actions.

Scale with OT-control

OT security and asset management require scale to drive efficiency. With that scale comes the risk of central resources unknowingly making inappropriate changes to running systems. The Verve Security Center enables clients to scale the analytics and reporting across all sites and assets into a single platform while allowing local OT teams to control the timing and testing of final actions on their control systems.

Application of modern endpoint security within sensitive OT environments

The Verve architecture and OT-specific endpoint security platform allows IT teams to achieve the same endpoint security aspirations in OT as they do in IT. The OT-XDR capabilities combined with Verve’s unique architecture means that it provides both distinctive security, while ensuring safe operations.