Network Stack Fuzzing: Amnesia 33 and MCUs (siblings to CPUs in most IACS devices)

It feels almost like yesterday where an organization or group of researchers released a batch of vulnerabilities for a network stack; in particular, URG11, Treck TCP/IP (Ripple20), and now “Amnesia:33”.

Well, the latter is another family of vulnerabilities that was released, coming in at a nice, odd total of 33 CVEs.  Similar to another release by another cyber security vendor who performed network fuzzing and code analysis of an Open-Source network stack, this set of vulnerabilities is far less implicative of the impacts that have been touted to have MASSIVE and MILLIONs of affected devices. Except it was failed to be grounded by the fact that this network stack code is generally used in small very-low resource-constrained microcontrollers (MCUs) for the most very basic Internet of Things (IoT) tasks.

However, the series of vulnerabilities are rated with a CVSS score of 9.8 (making it “critical” under SPECIFIC CIRCUMSTANCES IF PRESENT) and affects these open-source components/projects:  uIP-Contiki-OS, uIP-Contiki-NG, uIP, open-iscsi, picoTCP-NG, picoTCP, FNET, Nut/Net. This component is embedded within a product’s firmware and provides most of the software stack required for basic network connectivity to and from an affected embedded system over an Ethernet interface:

• It includes the following underlying protocols, but also provides functionality for other industrial protocols, which are stacked upon them (e.g., Modbus TCP/IP): IPv4, TCP/UDP, IPv6, DNS, and ICMPv6.
• It defines the state machines, requests, and responses for those protocols, maintaining active lists of established connections, and manages to create/close/close connections.
• It is sometimes present and largely affected by flaws that are the result of incorrectly parsing TCP/IP flags, poor memory management, and lack of data sanitization in protocol implementations.

BUT DON’T FRET! It will not be the end of the world if you have millions of affected devices largely because these devices perform only very specific functions, and they are greatly outnumbered by their more powerful brethren (e.g., a 32bit ARM CPU vs. a 32bit ARM MCU) that run Linux, VxWorks, who provide (reasonably or better) performance in Industrial Automation and Control System (IACS) and are far more likely to be in a sheltered position.

It could be argued that many IoT and IACS/ICS devices are directly connected to the Internet, and perhaps that is true, but I suspect there is more to the story. Here is why:

• Most industrial OEMs do not move “millions” of pieces of units of dedicated ICS equipment in most applications. They may for smaller things such as sensors, but those are often connected over a field bus, dedicated I/O cards, and accessed through devices such as PLCs or DCSs.  MCUs are usually in hard real-time conditions, and not used in data processing for the most part (exceptions exist, but is largely a costing problem).
• Most of the devices that use the stack described by the report are used in a situation where there is network media that makes raw access to the Ethernet and TCP/IP stacks difficult. For example, they may be used in a System on Chip (SoC) that may be routing IP traffic on top of a lower-level Radio Frequency (RF) protocol.
• Developing organizations may have identified these vulnerabilities using their own tools, patched them, or compiled them to have some of these features disabled, or rendered inert due to the configuration (e.g., the IPv6 stack is an option that may not be present by default).
• Devices with this stack may not be communicating over IP at all, either due to a physical configuration (aka the stack is there, but an Ethernet NIC is not present or used at all). A few public examples of this stack are some 500K “smart city lights” or parking meter installations that are likely talking using LoRa, 6LoWPAN, Zigbee, and NOT TCP.
• If by chance, millions of MCUs are sold, they are likely to be found in a dumb thermostat with a segment LCD screen and a few buttons. In fact, my Honeywell thermostat has a small Atmel MCU in it… who knows, maybe there is a PHY on it or even this scary stack.
• Many of these vulnerabilities may never reach these devices, or really allow much to be accomplished by compromising them. For example, in ContikiOS, you would need to have a device with a compiled in a shell or you may have to get far more creative to drop your own onto that system and pop it.  Or to not have a router in front of it, an ISP, and other network infrastructure making execution of several of the noted CVEs difficult.
• Of the affected stacks three are end-of-life and therefore, their applicability towards the millions of devices may also be tapered further, and so out of X downloads from those opensource initiatives.
• The actual score and applicability of these 33 vulnerabilities vary greatly. There might be 33 found in this single disclosure, but in an affected system, only one may be present or be a variable, and it may have a score FAR LESS THAN 9.8 (example – Harting’s or even Genetec far lower scoring disclosure for devices affected by this vulnerability).
• While knowing your risks is generally beneficial, I would be more concerned about my banking information being stolen, my emails being hacked, or my home NAS suffering a ransomware attack.

This is not to say there was no value in this research, and with the timely attacks of 2020 or the SolarWinds (#Soligate) supply chain breaches – this should be just another reminder for systems developers and OEMs to:

• Perform adequate protocol fuzzing using automated and freely available tooling (e.g., Scapy).
• Go back into time, and review adequate realistic vulnerabilities as discovered by my fellow esteemed industry comrade Adam Crain (seriously and this is not the first discussion on this topic).
• Execute destructive testing to discover your system’s responses/susceptibility that is out of normal operation.
• Scan your code using vulnerability scanners (statically, dynamically, somehow!)
• Never assume your components are secure or well-engineered (most vulnerabilities are due to gaps in product testing, and software engineering).
• If you are someone building upon a board support package (BSP) such as the ones provided by companies that produce reference designs – update the code and keep updating it. Alternatively, before investing in that platform, make sure it is a recent codebase!

If you are an asset owner:

• Include cyber security testing as a pre-requisite to the acquisition and verification by a third party before final acceptance.
• Require your deployment to have adequate cyber security Factory Acceptance Testing (CFAT) and Site Acceptance Testing (CSAT).
• Expect cyber security certification for embedded devices (e.g., ISA 62443 SL-T 2 not SL-T {0,1}).
• Recognize that vulnerability scoring metrics have flaws but are indeed an indicator that should be heeded according to a pre-defined decision tree. You need to apply your own modifiers or apply judgment with sufficiently capable help.
• But if you already have vulnerable assets or suspect that you do – begin by performing a detailed inventory assessment and take steps to confirm with OEMs/vendors that you are not susceptible. Information may be already released (e.g., such as the disclosure by Siemens who only has a few limited devices being affected).
• Update the device’s firmware IF your risk model suggests that it exceeds your organization’s security tolerance, it is legislated, your policies and governance recommend best practices, and it is within the purview of the organization’s patch/vulnerability program.
• And isolate and segment these devices adequately from a network perspective such that tricks such as modifying TCP MSS values are NOT possible. One option might be MSS clamping, denying IPv6, and using protocol sanity check features commonly provided by a proper firewall.

And if you are a researcher:

• Recognize that your research has value (and I approve of projects such as this), but marketing overemphasizing not only lowers its intrinsic importance but does cyber security industry a disservice when communicated to fearmonger.
• Realize asset owners are growing tired of the boy who cried wolf about cyber security claims beginning to speak. They speak in actual risks, and risk reductions – not raw potentials with a very low likelihood of being a target of a commodity weaponized exploit.
• Remember I’m still your digital friend and enjoyed reading the findings.

So regardless of the above advice, a pragmatic look on this topic would be to assume all embedded devices are vulnerable with numerous undisclosed issues, suffer from infrequent change, but often potentially execute a critical function in IACS environments and critical infrastructure (and no CVSS scoring mechanism could truly reflect YOUR organization out of the box).

That is not to say there should not be consequences to the OEMs and their suppliers due-diligence wise, but rather what is done is done. Instead of spreading Fear-Uncertainty-and-Doubt, let’s focus on the good. CISA posted the worst-case scoring, and reality looks far better for once in 2020 due to the nuances of vulnerability application.  But – a potential vulnerability does not equate to exploitability!

Vendors and FOSS project maintainers: go forth with lessons learned while taking steps to not repeat them (please).  Asset owners: let’s go protect some devices!

For more information, reach out to [email protected], review the mitigation notes on the CISA advisory, and check out our recent embedded devices webinar and white paper (see below).