The convergence of Operational Technology (OT) with Information Technology (IT), has exposed many challenges for the industrial space, including increased vulnerability to cyber attacks and network blindspots. Unfortunately, many companies are on the wrong side of a hack when they discover they do not have proper visibility into their OT systems, like they may enjoy with their IT infrastructure.
Also, as in IT environments, most industrial security and network monitoring tools are packet based. Engineers tend to run into some inherent challenges within this infrastructure on how to access those packets — SPAN ports may be available on OT switches but are prone to drop packets, duplications, may already be in use, or even some older legacy switches may not even have SPAN port options. They may also question how to incorporate TAPs in the infrastructure.
Once you have decided on a security strategy and what network monitoring and security tools your environment requires, laying down a visibility architecture requires figuring out the right cabling, connectors, mounting and packet capture devices needed. Here, we will review 7 of the most common visibility challenges in industrial environments and how to overcome them.
1) This is Not Your Traditional Network
From disparate systems and regulations to topology and DIN Rail mounting, it is clear this environment is very different. Balancing multiple systems and devices, from Supervisory control and data acquisition (SCADA), distributed control systems (DCS) to industrial automation and control systems (IACS), then add in many different topology concepts like Point-to-point, Bus, Star, Ring, Mesh, Tree, Hybrid and Daisy Chain, and you can see why architecting a visibility layer may get tricky or plain overlooked.
Within the network framework, we understand you’re not running 19-inch racks and you're not running AC power. The goal in this environment is to have as few moving parts as possible to minimize the risk of a cable coming unplugged or disrupting the network. Many industrial networks with industrial control panels use DIN Rails mounts, which are a mounting system used to secure or install electrical devices to the network and run DC power.
2) Following Standards and Regulations
Standards, certifications and regulations ensure that industrial processes are standardized and seamless, leading to proficient and safe environments. And the industrial space has many, including Standard developing organizations (SDO), such as the IEEE, which set specifications for implementing wireless LANs across several frequency bands, IEC (International Electrotechnical Commission) who enables the intelligent electronic devices in electrical substation automation systems to communicate, and PROFIBUS User Organization (PNO) who oversees certifications and standards related to PROFINET, PROFIBUS, and Field Device Integration (FDI).
The bottom line is that there are many networking requirements on scalability, convergence, security, performance, flexible topologies and low-Latency that remain key challenges in industrial applications. Some EU and U.S. security compliance requirements are now even standardizing which security tools or monitoring tools to use. And, of course, those tools are packet based.
3) Overcoming the Distance
Does your network cover a manufacturing plant, a refinery, a utility system or multiple locations? We know these aren’t the traditional business locations, where every floor has its wiring closet and you have your nice data center. These facilities can be massive. That is where distance considerations come into play when thinking about your visibility architecture.
We find one of the biggest contentions companies must deal with is restrictions on cabling lengths. While copper and fiber are both supported, copper device-to-device connections cannot exceed 100 meters. Though Copper ethernet has quickly become the standard and is widely used, the distance restrictions can easily become an issue in expansive networks. Fiber connections are much more accommodating for further distances, with a 2,000-meter limit—but fiber isn’t always an option depending on the environment.
4) Accessing Your Physical Challenges
Physical challenges aren’t just dealing with distance. Many engineers run into environmental issues and have to consider a variety of cable questions. Are you running an environment that is extremely cold or hot, has UV exposure, or possible crushing? Will your cables be exposed to oil, water immersion, moderate vibration, or chemicals on the factory floor? Will your cables be protected in a specific control room? These factors are important considerations when laying out your visibility architecture and determining how to access that data.
5) How Do You Access The Data?
The choice comes down to a network TAP or a SPAN port. Are you going to mirror a port from the switch or deploy a purpose-built device designed for traffic access? If SPAN ports are available on OT switches, they are prone to drop packets, duplications, and more than likely may already be in use. Even some older legacy switches may not even have SPAN port options. Aside from the well known issues SPAN ports propose, there are a couple of considerations specifically for the industrial environment.
First, many industrial routers and switches are unmanaged, do you need the links to your network and security tools to be unmanaged? SPAN ports are managed and often require re-configurations, most network TAPs, on the other hand, are “set it and forget it.”
Second, are you working with unidirectional gateways? We see people try to use SPAN ports in these situations and don't realize a SPAN port is bi-directional, which creates an unintended opportunity for hacking. In this situation, specifically designed Data Diode TAPs offer a unidirectional connection, providing the added security needed.
"SPANs can add overhead on a network device, and that SPAN port will often drop mirrored packets if the device gets too busy. Therefore, TAPs are a better option.” -EMA [Enterprise Management Associates]
6) Network Recovery Time During A Power Loss
Based on your visibility strategy in consideration of what systems, infrastructure, standards and regulations you have to adhere to, this factors into how you need to collect packets passively or with an active relay-based device.
The two main questions become, what is the level of failsafe technology that's needed in your environment and are you using Copper gigabyte? If you run into a power loss condition in a mission-critical environment, network recovery time is critically important. The best option in that scenario would be to use passive fiber or 10/100M Copper TAPs. These are completely passive devices and means that they can lose power and the link will remain up. If you are using Copper Ethernet, an active TAP solution, 100/1000M can be used in environments where if there is a power loss, the link will drop and then re-establish between 30-300 milliseconds.
7) Getting Your Wires Crossed
With so many connectivity options, operating systems, and bridging legacy equipment with security and performance monitoring tools many engineers run into, how do you connect various connector or media types? What do you do if your network analyzer runs copper gigabit and you need to connect a 100Base-FX link? There is no 100Base-FX NIC card for your security or performance monitoring device.
When architecting your visibility fabric, specialized network TAPs, like those offered by Garland Technology provide media conversion to easily solve those issues while providing full-duplex copies of the traffic through 100BASE-FX/LX, LC, ST fiber connections.
Solving Your Visibility Challenges
We understand that you are up against many challenges in an environment that can vary from the traditional network, like having to consider standards and regulations, and cover large expansive networks that entail many physical challenges. But the need for network visibility is real. When critical infrastructures are involved, companies can’t afford blindspots, dropped packets, traffic bottlenecks or suffer network downtime.
According to SANS 2019 State of OT/ICS Cybersecurity Survey “Visibility is critical for managing OT/ICS systems. According to survey respondents, increased visibility into control system cyber assets and configurations is the top initiative organizations are budgeting for in the next 18 months.”
Deploying network TAPs throughout the Industrial Ethernet framework ensures uptime and eliminates the packet delivery issues that SPAN/Mirror ports inevitably introduce. Garland Technology also has an assortment of industrial based TAP and accessories, including DIN rail network TAPs, DC-DC power converters, screw power lock connectors, media conversion TAPs and Data Diode TAPs - all to provide extra assurance to overcome the connectivity and environmental challenges you may face.
Not sure where to start? Set a meeting with one of our network engineers for a no obligation, Visio network design session!