Visibility Solutions

Garland Technology is committed to educating the benefits of having a strong foundation of network visibility and access. By providing this insight we protect the security of data across your network and beyond.

Resources

Garland Technology's resource library offers free use of white papers, eBooks, use cases, infographics, data sheets, video demos and more.

Blog

The TAP into Technology blog provides the latest news and insights on network access and visibility, including: network security, network monitoring and appliance connectivity and guest blogs from Industry experts and technology partners

Partners

Our extensive technology partnership ecosystem solves critical problems when it comes to network security, monitoring, application analysis, forensics and packet inspection.

Company

Garland Technology is dedicated to high standards in quality and reliability, while delivering the greatest economical solutions for enterprise, service providers, and government agencies worldwide.

Contact

Whether you are ready to make a network TAP your foundation of visibility or just have questions, please contact us. Ask us about the Garland Difference!

Blogheader image

TAP Into Technology

Leading the Way in Network Technology

Are my Packets Lying? - 4 Things to Look for in Packet Traces

Posted by Chris Greer | 6/13/17 8:00 AM

Packets don’t lie – well, most of the time.

They tell the truth unless they have been captured incorrectly. In those cases, packets can tell bold-faced lies.

When digging through trace files, we can come upon symptoms in the packets that may raise an eyebrow. These are events that look strange on the surface and may even divert our troubleshooting focus for a time. In fact, some of these issues have misdirected engineers for hours, if not days, causing them to chase down issues and events that simply did not exist on the wire.

Most of these examples can be avoided simply by capturing the packets from a network test access point (TAP) rather than on the machine generating the traffic. (Come on, you know you have needed a TAP for a while! Just spring for one and capture correctly next time.)

Very Large Packets

Screen Shot 2017-06-06 at 3.17.08 PM.png

For the most part, packets should not be larger than the Ethernet maximum of 1518 bytes, or whatever the link MTU is set to. That is unless we are using 802.1Q tags or are in a jumbo frame environment.

How is it possible to have packets that are larger than the Ethernet maximum? Simply put, we are capturing them before they are broken up by the NIC. Many TCP/IP stacks these days use TCP Segmentation Offloading, which delegates the burden of segmenting the packets to the NIC. The WinPcap or Libpcap driver captures the packets before this process happens, so some of the packets can look far too big to be legal. If the same activity was captured on the wire, the large frames would have been broken into several smaller ones for transport.

Download TAP vs SPAN now!

Zero Delta Times

Screen Shot 2017-06-06 at 3.17.16 PM.png

Zero delta times mean that there is no measured time between packets. When these packets entered the capture device, they were timestamped the same as the most recent one with a measureable delta time. The ingress timestamping on the capture device could not keep up with the packet load. If these packets were captured with a TAP external to the server, we would likely see correct timestamping.

Previous Packet Not Captured

This warning is shown because Wireshark interpreted a gap in the TCP stream. It can determine from the sequenced numbers that a packet went missing. Sometimes this is legitimately due to upstream packet loss. However, it can also be a symptom that the analyzer or SPAN dropped the packet because it couldn’t keep up with the load.

TAPTip-Icon2-LR.pngTip – after this warning, look for a series of duplicate ACK packets, then an Out-Of-Order packet. This indicates that a packet was indeed lost and needed to be retransmitted. If you don’t see a retransmission or out-of-order packet, then the analyzer or SPAN probably could not keep up with the data stream. The packet was there on the wire, but we didn’t see it.

TCP ACKed Unseen Segment

In this case, we see an acknowledgement for a data packet that was not captured. The data packet could have taken a different path, or the capture device simply did not pick it up.

Recently, I have seen these events from trace files that have been captured from switches, routers, and firewalls. Since capturing traffic is a lower priority than forwarding (thank goodness!) the device simply missed some of the frames in the stream. Since we saw the acknowledgement, we know that the packet did make it to its destination.

For the most part, packets tell the truth. They can lead us to the root cause of our network and application problems. Since they present such clear and detailed data, it is very important that we capture them as close to the wire as possible. This means capturing them in transit and not on the server itself. This will help us avoid wasting time on the false negatives.

If you want to learn more about Real Network Visualtion Consideration for Professionals, download our free white paper, TAP vs SPAN. 

Topics: Network TAPs, TAPs vs SPAN, Wireshark

Written by Chris Greer

Chris focuses on assisting IT professionals in resolving the root cause of network and application performance problems. This is primarily done through use of a protocol analyzer reading and interpreting trace files. Additionally, I teach courses in network analysis and troubleshooting, Wireshark, and several vendor analysis products. Chris supports clients in several regions of the USA, Latin America, and Caribbean.

Sign up for blog updates