WISP Network Bottlenecks
Most of the Quality of Experience (QoE) issues we see are due to some form of network bottlenecks. These are found at access points, backhauls, individual subscribers, routers, or switches. In our experience, bottlenecks either add latency or cause packet loss and impact subscriber QoE.
There are two key ways to deal with network bottlenecks. The simple way is to just drop off any packet that doesn’t make it. This is the least graceful way to deal with a bottleneck, but that’s often what happens.
For example, if you have network congestion on a certain pipe but no shaper in front of it, that’s exactly the behavior you’ll see. This causes very high packet loss.
The main issue with this is that dropping packets doesn’t give TCP a chance to slow down and start behaving the way it’s supposed to. Also, as TCP notices these dropped packets, you get all kinds of re-transmits. This also adds extra traffic on your network and makes things more difficult.
Examples of this behavior we’ve seen are at networks where:
- Cambium 450s are used for plan enforcement
- Backhaul congestion exists
- Links/ports can’t keep up with the traffic requirements
Network Bottlenecks Create Buffering
The other mechanism is all-around buffering. Basically, as you have the network bottleneck and all the traffic can’t make it through, the shaper will simply buffer some of the traffic. This also adds latency and causes TCP to back off. This works effectively if you have a shaper that isn’t too deep and just buffers it a little bit to smooth out “bursty” behavior. However, if this buffer gets too big, it’ll add way too much latency, thereby impacting QoE.
As a WISP, you should aim to fine tune this piece. Different kind of shapers behave differently. Examples we’ve seen across wireless networks where bottlenecks cause buffering are:
- Access point congestion, adding up to hundreds of ms of latency
- Shapers
- Interference on your APs
- On the Cambium side, if you have an EPMP for plan enforcement, it’ll add too much latency
- Software mismatch, and more
The Problems Caused by Latency and Packet Loss
The common themes in this discussion are latency and packet loss. These are both very critical to your customer’s QoE. Higher latency results in poor VoIP calls, real-time gaming, and also unresponsive or slow browsing.
To have a good VoIP call, for example, you need a round trip time of under 150 ms. That’s from phone to phone and back again, all the way across your network, the internet, and also the other party’s network. As a WISP, you’re only controlling half of that. As a result, you’ll want to stay under 75 ms to give the rest of the network a chance to add latency but still have good subscriber QoE.
On the packet loss side, lost packets get retransmitted. The overall effect beyond that is similar to latency but it also adds jitter and extra network traffic. We’ve seen that a 2-4% range for TCP retransmit is pretty normal. If this goes over 5%, you know that your network needs some careful examination and fixes.
