The convergence of IT and OT in the context of Industry 4.0 has led to a crowded market of security solutions targeting the shop floor on different levels. While the security of the legacy IT systems was part of the initial planning of those systems, the industry now faces the challenge to integrate security solutions in legacy OT systems.
Figure 1: Layers of a typical automation system using Profinet
At least for the lower levels of the Industry 4.0 infrastructure, (control and device level) security solution vendors tend to use the mirror/span ports offered by the network switches to integrate their solutions into the infrastructure.
Here I will explore why those mirror ports shouldn't be used to build security solutions. Based on the hardware commonly used in Industrial Networks, we will specifically look at:
- How it all works
- Why a mirror port will drop frames even if a link is not saturated
>> Download now: Learn why SPAN Ports should not be used in industrial security solutions
Saturation Overload
I came across some research data Garland posted from Packet Pioneer that said a mirror port will drop up to 8% of the frames.
I took a deep breath because an 8% loss is a lot, especially if you are going to build security applications on top of the mirrored network traffic from systems using Profinet in different flavors (configuration, real-time and alarm IO).
I continued reading and calmed down, in their research, they used iperf to saturate a link that was mirrored at the same time. Saturation means the system is in an overload condition, so drops are acceptable. And every network engineer will agree, that using more than 80% of the bandwidth of an Ethernet link will result in weird behavior at some point.
But I couldn’t get past these questions, ‘What happens, if only small frames are transmitted?’ The average frame size in industrial networks is ~130 bytes, (calculated from a sampling of shop floor traces containing mostly Profinet) but the aforementioned test was done using 1500 byte frames (iperf usually uses TCP and that will send the data in chunks of MTU size). ‘What if we use more than one link?’ On the process level usually one controller talks to a number of IO-devices via one switch.
Figure 2: Process level - Profinet master and IO-Devices
In the context of industrial automation systems a test setup using a one-to-one connection running 1500 byte frames does not reflect a common scenario. And regarding automation systems, what if the traffic on those multiple links shows different bandwidth patterns?
Test Setup
Keeping those questions in mind I built a small test setup based on the open source network traffic generator and analyzer Ostinato. This tool offers a nice interface to create well defined streams of Ethernet frames for multiple network interfaces while intercepting traffic at the same time.
This is not a hardware frame generator, but capable of handling multiple 100 Mbps streams with relative low-tech network interface cards. 100 Mbps is absolutely sufficient for running tests against network switches commonly used on the lower levels in a shop floor automation environment, as this is the usual network speed used there.
Want to see the test details and results? Read Thomas' tech note on Full Duplex Capture in Industrial Networks to learn the best practices for using network TAPs in Industry 4.0 security applications.
