.png?width=40&height=40&name=search%201%20(1).png){kind=small}
I spent many years as a technical support engineer for a Telecommunications company that did voice and data services in third-tier markets. We would be called in to troubleshoot any performance issues with the equipment.
Our customers were small and medium-sized businesses in small towns scattered around the east coast, like insurance and doctor offices. Typically, their IT set-up consisted of a home router and our modem and VoIP phones. Some of the businesses would manage the data side of their networks but outsource their IT for firewalls and routers. Most of these companies were running on a shoestring budget and relied on our support for their success.
So I would be working with a field engineer who would go out to these businesses and we would try to pinpoint where the issues were. We would be using Wireshark to analyze the packet captures and protocols and have to rely on cheap store-bought switches or old hubs to access the packets.
Working as Garland Technology's Tech Support Manager, and after being able to work with devices like Garland’s FieldTAP, I now realize why this should be in every troubleshooting kit for all field engineers. With a Garland FieldTAP, we would have been able to pinpoint issues and resolve cases in a fraction of the time, standardizing the company's support efficiency, ultimately leading to happier customers and potentially more business.
In a typical case, the field engineer would arrive at the location and we wouldn't be certain where the problem would be. So the first goal would be to locate the issue – was it before the modem, between the modem and the router, between the modem and the VoIP phone? Or could the problem be between the router and the firewall or after the router, to the VoIP phone?
The proper strategy would be to run packet captures between each appliance and see if we received packets. The segment without packets was usually our trouble spot. Sounds easy enough, but this was always our first major challenge. The field engineers for many companies would have to supply their own devices, which ended up being a store-bought 4 or 8 port switch, or an old ‘hub’ if we were lucky, to try and access the packets. Neither is an ideal device to be troubleshooting with.
I could log in remotely into my switch from the central location and try to get a reply in the field and then troubleshoot where it was getting lost – I could send a ping, and that would tell me if it was a valid IP address and that the network sees it. Then once we power cycle the phone, now I'm looking for protocols and then application layer stuff.
Since many times we didn’t have proper packet capture options, we had to work blind, improvising a solution. When packet capture attempts failed, the field engineers would often take a spare modem, routers, phone, and cables and just swap out the equipment because that was the only option to pinpoint the issue.
The next challenge we would run into using these ‘store-bought’ switches to capture packets for Wireshark is that they simply didn’t work. These switches forward packets to a particular port based on the Mac address. And if you're inserting a device between the modem and the router or the router and the firewall, or the router and the VoIP phone, and if they're routing based on Mac address – your Wireshark won't see anything.
Another challenge was overcoming power issues. You had to be close to an AC outlet, or you’d have to run really long ethernet cables. These switches don't forward the power over ethernet (PoE), so the phones would go down while troubleshooting. A lot of these companies would have their voice phone hooked up to the jack, and would hook their computers up to the second jack, which saves money on infrastructure. But when you break that PoE connection, the phone goes down.
Another challenge with troubleshooting was dealing with various location restrictions. A lot of times when you go out to these places, you're in a small, dark telco closet. And you're struggling with where to put your PC, are my cables long enough? How do I get power to my switch or hub?
Then there's the issue of high turnover with Field Engineers. Simply put, they get frustrated. We’d be sending them out to do a job without the necessary tools they needed. They’d work a couple of years for the training and experience and then move on.
Many of these challenges would have been solved if our team simply had a network TAP or specifically Garland’s FieldTAP. These devices are purpose-built to make a network engineer’s job easier.
One of the main benefits of using the FieldTAP is it passes PoE and is powered off of the USB 3.0 connection. You just insert it, the phone comes up and then you hook up your laptop to the USB port, and you can capture packets using Wireshark. This eliminates the challenge of power access.
Network TAPs of course are best practices for accessing packet data, providing complete full-duplex copies without dropping packets, introducing delay, or altering the data. This is important when you're capturing off your laptop using Wireshark because now you can see both directions of the packet flow. A hub may be only half-duplex, a switch may only route to MAC addresses and as we’ve discussed troubleshooting a VoIP can stop packets at the phone.
The small form factor of the FieldTAP allows an engineer to throw it in their bag or pocket, and it’s plug and play, ready to copy packets. And even more, there are variations of the FieldTAP that allow media conversion, allowing you to connect various fiber options and copper together.
Some engineers I've talked to refer to it as the swiss army knife of packet visibility. Because they are so easy to use and should be a standard issue for any troubleshooting team. The FieldTAP will shorten troubleshooting times and increase packet capture effectiveness, not only alleviating field engineer frustrations but also improving customer and end-user satisfaction. Because as we always say, ‘you can’t effectively troubleshoot blind.’
Looking to add TAP visibility to your Troubleshooting kit, but not sure where to start? Join us for a brief network Design-IT evaluation or demo. No obligation - it’s what we love to do.
