Quantifying QoE: How to Measure & Improve the Subscriber Experience

Quantifying QoE

We may have mentioned once or twice the importance of providing your subscribers with a great quality of experience (QoE) 😉 Delivering superior QoE is a great way to retain your customers, get positive reviews and word-of-mouth referrals, while also reducing support calls and churn. This in turn can help your operation grow, even in an increasingly competitive market.

Optimizing for great QoE is something we’ve been beating the drum about for some time because, well, it’s literally what Preseem is designed to help you accomplish. Our goal is to help local and regional operators compete against big national providers by delivering a superior experience to their subscribers.

If you’ve been following us for a while, then you’ll be well aware of our message that the better QoE you provide, the happier your customers will be.

But, you might be wondering, how exactly do you go about quantifying that? In other words, how can you actually tell if you’re providing good QoE, or if it’s improving over time? What does “good QoE” really mean? And how do you stack up against your fellow ISPs?

Well, you’re in luck because that’s what this blog is here to explain!

How to Measure Quality of Experience

For local and regional operators, QoE is all about understanding and improving the service delivered to customers. First, however, you have to be able to measure QoE in a way that’s truly reflective of the customer experience.

Traditional network monitoring systems can tell you the overall health of individual network elements—and that’s absolutely necessary—but they don’t really tell you very much about subscriber QoE.

At Preseem, we measure QoE by collecting metrics directly from subscriber traffic and network elements like access points. We then tie those metrics into the network topology, so that our customers can use those insights to proactively improve network performance and the subscriber experience.

For example, our QoE metrics can tell you which of your APs have issues or are over capacity, so that you know which ones you need to target for an immediate fix.

So how does it work? We measure TCP Round Trip Time, which is the time it takes a packet to travel from the Preseem appliance, through the backhaul and access points, down to the subscriber’s devices, and back again. Preseem continues measuring these trips constantly, to the tune of billions per day across our client base.

From these measurements, metrics are extracted and charts produced that quantify latency, retransmits, throughput, packet rates, etc. These can help identify contention/bottlenecks, overloaded backhauls or APs, poorly-performing equipment, even in-home Wi-Fi issues.

Because we measure traffic directly from the customer, it creates a complete picture of the experience being delivered—from our inline element all the way to a device being used in the customer’s home. This provides deeper insights than you might see in simple QoE or AP monitoring systems, and ultimately allows our customers to allocate their troubleshooting resources more effectively and efficiently.

How to Quantify QoE

So that’s how to measure QoE, but what about the meaning of those measurements? Let’s look at some of the actual metrics this process produces that quantify QoE, and how these numbers can help you improve your network.

AP and CPE Radio Metrics

Preseem collects and stores many AP and CPE Radio metrics that impact QoE. These include RF parameters, RSSI, modulation, frequency, channel width, and more. We then normalize these across all vendors and models, so that your team can see everything in one place and not have to ‘stare and compare’ in multiple systems.

These metrics are retained for two years and help empower your support team to investigate QoE issues without necessarily having to be an expert on each vendor’s tools. The two-year retention lets you compare current to past RF connections, so that you can easily see what’s changed or where any issues may have occurred in the past.

CPE Radio Scores

CPE Radio Scores are designed to help you identify where any RF issues exist in your network and make it easier to prioritize improvements. We take real-world performance information from hundreds of ISPs using Preseem and group them by model and configuration. We then combine this with the behavior of your specific subscribers and CPEs to create a simple 0-10 score.

This score means that—per AP or across your entire network—you can identify which subscribers are doing the most damage to AP capacity and negatively affecting the QoE of other customers. You’ll also see which subscribers have the worst RF conditions. In either scenario, you can prioritize addressing those to improve QoE for everyone on the network.

Access Point Scores

Much like CPE scores, the AP score combines performance data from hundreds of providers with the actual behavior of your specific APs to produce a simple score. This 0-10 score can tell you which APs have the worst RF, so that you can proactively fix it before customers start to notice. It also identifies your most overloaded APs so that you can address RF conditions for the worst subscribers to improve airtime, or decide whether you need to split or upgrade.

AP Subscriber Capacity

This score combines AP airtime and RF conditions with the behavior of your subscribers at peak busy hour to come up with a capacity number, with subscriber count as the unit. This shows you in a very simple way where you’re overloaded or where you have room to add new customers in a given area.

Track What’s Trending

As well as being able to see quantifiable QoE metrics that help you make key network decisions, it’s also helpful to be able to see which way things are trending. Preseem includes easy-to-understand trend indicators that show which direction your Access Point and CPE Radio scores are heading. This way, your team has the tools to quickly identify which scores are trending downward so they can address the issue proactively.

Apply QoE Analysis to Help Your ISP Grow

One of the nice things about having all this reliable QoE data on hand is that you can start to use it to better understand you customers and improve your business. Here’s a few ways in which QoE analysis can help you grow.

Improve Your Marketing

By knowing which access points still have room to add more subscribers, your marketing team can proactively target specific neighborhoods or regions covered by those APs. This is a more efficient use of your marketing dollars, and removes guesswork and ‘gut-instinct’ decision-making. Also, providing great QoE can help get you the best and most cost-effective kind of marketing—word-of-mouth referrals from satisfied customers.

Optimize Your Business Planning and ROI

By using quantifiable QoE metrics to identify which APs or towers are having issues, your techs can prioritize where to spend their time most efficiently and you can identify which areas of your network need investment.

Also, by using QoE metrics to improve the subscriber experience and fix issues before they affect your customers, you’re freeing up time and revenue for your staff to focus on proactive network improvements, as opposed to spending their time constantly putting out fires or going out on costly truck rolls.

Get to the Root Cause of Issues Faster

It’s not really practical for local and regional operators to try and find RF or QoE issues by manually looking at each AP and CPE Radio. As a result, most operators are reactive to QoE problems and only check the radios when a customer complains. Not to mention there just aren’t that many RF experts around to help interpret highly technical and specialized issues.

Your L1/L2 support and resident experts can get to the root cause of issues quickly with quantified QoE metrics. For example, by being able to see when RF quality changed, you’ll have a better chance of figuring out why it changed (e.g. maybe there was high winds that day).

They can also be used to help you grow—for example, by knowing which specific customer’s RF conditions you need to improve to free up the most airtime, you’ll not only be able to help that customer but sell more plans on the AP as a result!

If you’d like more information on how your ISP can start quantifying your QoE and providing your subscribers with a better experience, get in touch with us and we’ll be happy to show you how we can help.