Product Usage Analytics: The Product Manager’s Guide to Uncovering Hidden Usability Issues [INFOGRAPHIC]

Looking to improve your product experience? We all are! Product usage analytics might just be the answer. By collecting and analyzing data to understand how users interact with your product, you can uncover hidden usability issues that may be hindering their experience. In this article, we’ll discuss 5 tips that can help you discover those issues and take action to improve your product development flow. Without further ado, let’s dive into the product usage analytics process!

What is product usage analytics and how do you collect the data?

This broad term includes various types of data derived from different touchpoints across the user journey. Product usage analytics defines the scope of an analysis of how users navigate the product, what features they use, what problems they encounter, and ultimately – what value they derive from it. 

With this type of analysis, product managers can answer a lot of questions that keep them awake at night:

• How do adoption or retention rates vary across different user segments?
• What is the level of user retention or stickiness of the product?
• How effective are the engagement strategies and campaigns used?
• What is the user workflow while using the product?
• Which features of the product are most popular among users?
• What are the friction points or issues that users face while using the product?
• What are the user engagement levels of the product?
• Which user types heavily use the product?

Methods of collecting product usage data

The most common way of collecting product data back in the day before the Internet was through customer journey surveys and focus groups. While still widely used, today we have a new way to measure a product’s impact on customers in the digital sphere – digital experience analytics. A combination of UX analytics, product analytics, and core web analytics, these tools combine quantitative and qualitative data to paint the user journey in vivid detail. 

Some analytics tools use manual instrumentation which requires engineers to tag every major (and minor) event that you want to track. On the other hand, tools that rely on tagless autocapture require only a simple code snippet to work – and they track every user interaction in real time. 

Data collection tools include dashboards with metrics such as session duration and bounce rate, funnels that visualize drop-off points, A/B testing platforms, session replay, customer surveys that can quantify user satisfaction, and many more. 

Are you getting the right data?

Apart from having to decide what digital experience analytics tools to use, product managers need to know how to use them to reach their goals. This process begins by setting the right goals (duh!) and mapping the right KPIs and metrics on them. Finally, when analyzing the data, you need to make sure that it is statistically significant.

How to use product usage analytics tools in your day-to-day tasks: some examples with SessionStack

We hear you, you don’t like Excel. Well, it’s not your fault, it just doesn’t spark joy. Fortunately, modern product analytics tools rely on visualization to show you what you need to know. As highly visually motivated as we are as a species, this certainly evokes a sigh of relief. 

Let’s take a look at SessionStack, for example. In our Segments dashboard, you can see the sessions of your users on the left and a bunch of pretty & useful graphs on the right. These diagrams can give you a bird’s eye view of what’s happening in your product – session data, where your users are clicking (Most Clicked Elements), and whether they are having trouble somewhere (Rage and Dead Clicks). 

In this part of the platform, you can also build detailed segments using User, Session, and Event-based filters. This helps you gain a better understanding of your customers segment by segment, and thus deliver better user experiences for all of them. For example, you may notice that your product is more sticky for users who experience your product on a specific OS or are from a specific country. 

On the Funnels page, you can build a simple (or complex, up to you) funnel to spot how things are going on the conversions front. Many people entering the funnel, only a few converting? You can now spot the step that drives the biggest drop-off. A page is converting better than the rest? Reverse engineer to understand why it works better and replicate the success!

Equipped with this knowledge, you can now proceed to the real deal – Session Replay. What’s better than experiencing the product as if you were in your users’ shoes? Hit the Play button of a desired session (hint: you can select to view only problematic ones to save time) and see the user session in a pixel-perfect replay.

You will be able to spot the exact moment when they encountered a problem – this alone will save you tons of time replicating bugs. But it also gives you the most effective tool in your product usage analytics stack: the ability to empathize with your customers, understand what drives their behavior, and deliver solutions that work for them every single time. Add to that real-time co-browsing and you’re set for success!

How to discover your users’ hidden usability issues: 5 tips

What you believe your users’ pain points are and what they really are might be two totally different things. Sometimes, we are guided by heavily biased assumptions and it takes a lot of willful effort to take a step back and reevaluate. Data can be the eye-opener you’re looking for. So, put on your reading glasses, and let’s unpack the gist of this article – how you can discover the true usability issues of your product with the help of product usage analytics.

Tip 1: Analyze user engagement

User engagement is used to measure how much users are interested in and interacting with your product. By keeping an eye on user engagement, you can figure out which features or aspects of your product are the most popular and valuable to users, as well as which ones might be causing them to feel confused or frustrated. 

Some metrics you can use to track user engagement include:

• Time spent on specific features: If users are spending a lot of time on a particular feature, it may indicate that they find it valuable or are having trouble using it.  Conversely, if users are spending very little time on a feature, it could be a sign that it’s not very useful or intuitive. Double check the reason with the help of Session Replay. 
• Number of clicks: Tracking the number of clicks users make can help you identify which parts of your product are most frequently accessed or used. Or, if users are clicking on something repeatedly (Dead Clicks or Rage Clicks), it could indicate that they’re having trouble accomplishing a specific action.
• Conversion rates: If your product has a specific goal, such as signing up for a service or making a purchase, tracking conversion rates can help you identify where users are dropping off in the process.

Example: 

You notice that users are spending a lot of time on the checkout page but not completing their purchases. This could indicate that there’s a problem with the checkout process that needs to be addressed. The possible culprit might be a specific payment option but you may need need to replay some affected sessions to find out the truth.

Tip 2: Monitor user feedback

Listening to what users have to say about your product can give you some pretty useful insights into the usability issues they’re experiencing. You can get user feedback in a bunch of different ways, like surveys, feedback forms, and reviews. 

Some metrics to keep an eye on when it comes to user feedback include:

• Number of feedback submissions: Tracking the number of feedback submissions can give you an overall sense of how users feel about your product (don’t let your support team suffer, be proactive in resolving arising issues!).
• Sentiment analysis: Behavioral analytics tools that analyze user sentiment can help you identify which aspects of the product create the most frustration and require immediate attention so you can prioritize better.

Example: 

You receive multiple feedback submissions about a particular feature being difficult to use or confusing – it may be worth investigating further to see if there’s a way to improve it.

Tip 3: Track the user journey

We refer to a user journey to describe the path that users follow to achieve their goals with your product. If you keep track of this journey (and its variations), you can pinpoint the places where they are getting stuck or frustrated and dropping off. 

Some metrics you can use to track the user journey are:

• Time to complete a task: If users are taking a long time to complete a task, it could indicate that there are obstacles or usability issues in the process. 
• Abandonment rates: Tracking abandonment rates at various points in the user journey can help you identify where users are getting stuck or frustrated. Funnels are especially handy with this type of product usage analysis.
• Click-through rates: Analyzing click-through rates (CTR) can help you identify which points in the user journey are the most important so you can allocate more resources to improving them.

Example:

You notice that users are frequently abandoning their shopping carts before completing a purchase. This could indicate that there’s a problem with the checkout process that needs to be addressed.

Tip 4: Monitor user retention

User retention is defined by the number of users who stick around and keep using your product over time. If you keep an eye on user retention, you can see whether people are really finding your product valuable, or if there are usability issues that are causing them to jump ship. 

Some metrics you might want to track for user retention are:

• Churn rate: Tracking the churn rate can help you identify how many users are leaving your product over a given period of time. An increase in this metric might signify problems with the product, while a decrease can mean you’re doing a great job at improving your product – keep it up!
• Time between logins: If users are taking a long time between logins, it could indicate that they’re losing interest in your product or finding it less valuable over time. Now, the typical time between logins might vary between different products, but you can use industry benchmarks as a starting point and keep a close eye on this metric to spot irregularities.
• User engagement over time: this metric can help you identify whether users are becoming more or less engaged with your product over time. Great UX journeys immerse the users and keep them engaged, while bad UX can spell a disaster. 

Example:

A sudden drop in user retention or engagement could indicate that there’s a major technical issue with your product that needs to be addressed.

Tip 5: Compare user behavior across different segments

Doing product analysis using data sliced up in novel ways can reveal major opportunities. Comparing user behavior across different segments can help you identify unexplored usage patterns that may be specific to certain groups of users. 

Some examples of segments to compare include:

• New vs. returning users: New users may have different pain points than returning users who are more familiar with your product. Monthly active users vs daily active users is another useful type of segmentation.
• Mobile app users vs desktop users: Comparing these two segments might help you pinpoint differences in the way users interact with your product depending on the technology they are using. This might prompt the redesign of some features to better suit the mobile UX, for example. 
• Operating system X vs operating system Y: Some people love Apple’s ecosystem, some prefer Windows, and then there are the die-hard Linux fans. Each of them might be interacting with your product in a different way, encountering different usability issues. This type of user segmentation will help you gain a better understanding of the users in these groups and serve them better solutions.

Example: 

You notice that users using your mobile app are consistently abandoning your product at a certain point in the user journey – but desktop users aren’t. That could indicate a problem that is specific to that group of users. Addressing it will help you quickly improve the conversion rate of the mobile application.

To sum it up…

There you have it – a short introduction to product usage analytics with 5 actionable tips on how to quickly spot usability issues. All you have to do after you have the data is prioritize improvements and hand over the implementation to the product teams. Sounds simple, right? Yeah, right. We hope that we have at least shed some light on the sometimes confusing user experience optimization process.