This is the second post of our three-part Video Metrics Series.

In our previous post of this series, we discussed that video playback metrics can be divided into three distinct but related categories:

• Content/Business Objective Metrics
• QoE (Quality of Experience) Metrics
• QoS (Quality of Service) Metrics

Part one of the series discussed Content & Business objective metrics. This post will discuss QoE metrics, what you should track, and why.

QoE (Quality of Experience) Metrics

QoE metrics typically refer to subjective, user-oriented metrics. They indicate the overall acceptability of the application/service as it is subjectively perceived by the end user. They are often considered in relation to QoS (Quality of Service) metrics to bridge objective system performance metrics with the perception of the service from a user experience viewpoint. The below QoE metrics are imperative for content providers to measure.

Video/Picture Quality

Video and picture quality can be quantified (average bitrate, low-percentile bitrate, etc.), but the perceived quality by the user is going to carry more weight in overall QoE. If you understand what your users consider to be higher subjective quality, you can improve their experience with your service.

For example, for services using adaptive bitrate streaming, research suggests that users prefer a stable quality stream with fewer quality transitions – even at the expense of an overall higher bitrate. Furthermore, video quality perception is shown to be asymmetrical, with users more critical of quality degradations and less rewarding of increased quality.

Likelihood Of Rebuffer

Buffering issues (also called Stalling) result in viewer drop-off and decline in engagement with content. The more likely re-buffering is to occur and the more frequently it happens, the worse the Quality of Experience. The likelihood is affected by a variety of factors, including the environment of available bandwidth and the quality of the video. Content providers need to be mindful of delivery – for example, aggressively upshifting to higher video quality as the program progresses will increase the likelihood of a rebuffer mid-program, which can increase user drop-off.

Speed Of Startup

Speed of startup describes the amount of time it takes for video playback to begin. This metric can be tricky, due to its relationship with buffering. On the one hand, slow initial playback leads directly to viewer drop-off and a decline in engagement. On the other, high speed of startup allows users to access content more quickly but increases the likelihood of rebuffer when the viewer is watching, which has also been shown to increase drop-off and impact engagement. That said, viewers are typically open to longer startup times (to a point) for better viewing experience, so striking the right balance between the two is critical.

Product Metrics

There are a number of metrics you can track (on top of the QoE ones mentioned above) that can help you inform the evolution of or updates to your product. Typically, these will include application-level metrics that focus on how well features of the product are performing, for example:

• Visit/play history of particular content
• Click through rates on related videos and content suggestions
• Engagement with particular features/functions
• User feedback

Keeping track of how users are engaging with not just your content, but the product itself, can help you make product decisions that will make future iterations of your product more successful.

It’s important to keep in mind that QoE metrics are typically subjective and perceptual, but crucial since they provide insight into how the user judges the value of your product. In order to gain a full understanding of how well the product is performing – both from a delivery and user experience perspective – you need to collect QoS (Quality of Service) metrics on top of QoE metrics. We will discuss QoS metrics in the final post of our Video Metrics Series.