When you have completed the usability tests, it's time to get started with the analysis. In this article we zoom in deeper into analysing usability tests and help you with conducting your own analysis. 

1 - Keep the research objectives top of mind

To ensure that you maintain focus, you want to ensure that you have the goals of your user research top of mind. You are about to go through a large amount of data so you want to make sure you start the right way. Therefore, read the research objectives again and remind yourself what the focus areas are.

We often see that these focus areas correspond with the individual tasks in the usability test. For example, finding a contact number, requesting a quote or completing the ordering process.

2 - Identify usability issues

When you have refreshed your memory with the objectives of the research, it is time to analyze the usability tests. Make sure to note every obstacle, every frustration, every suggestion, and every negative or positive emotion.

It is advisable to work with this in a structured manner. In most cases it is sufficient to structure your notes by doing the following:

• Use a unique ID per note
• A short clear description of the problem
• Indication of the task where the problem occurred
• Indication of where the respondent encountered the problem (homepage, product page, order page, etc.)

When you put that together in a table you get something like this (based on the book Quantifying the User Experience)

Using the above template you can make a clear and structured overview of the information that you have distilled from the user tests. Now that it's clear what users are running into, the next step is to determine the importance of resolving these usability issues.

Not every problem has the same impact and to determine which problems have the highest priority, you need to take the following factors into account:

• How important the functionality is for the company and the end user (critical score)
• How many respondents experienced the problem (frequency ratio)
• The extent to which the problem influenced the respondent (impact score)

In a step-by-step plan it looks like this:

1. Determine the critical score. Assess how important this functionality is to the business and the end user. Use a scale from 1 to 5
2. Calculate the frequency score. To do this, divide the number of respondents who have experienced this problem by the total number of respondents who have taken the usability test. You will then get a score between 0 and 1.
3. Determine the impact score. Use the criteria below:
   • 5 (Blocker) Problem prevents the user from completing the task
   • 3 (Major problem) Problem causes frustration and delay
   • 2 (Minor problem) Problem has minimal impact
   • 1 (Suggestion) Suggestion of the respondent

Calculate the priority score. You do this by multiplying the critical, frequency and impact score. Based on the above, we can update the table from step two, which looks like this:

While most of the analysis is qualitative in nature, there are a number of quantitative usability metrics that you definitely don't want to miss on. 

Task completion rate or success rate

The success rate, also known as the task completion rate or task success rate, is one of the most well-known usability metrics. When the task that the respondent has clearly formulated and has a measurable end goal (for example placing an order), the success rate shows how well users can handle the product.

The success rate is the ratio of the number of users who have successfully performed the tasks of the usability test. You calculate the success rate by dividing the total number of successful tasks of all respondents by the total number of tasks performed by all respondents.

Time to task completion rate

The time to task completion provides insight into how much time respondents (on average) needed to complete a task. It helps to set objectives prior to the usability test, so that you can compare your expectations versus reality. 

In general, the longer a user needs, the more complex the task or process. You calculate the time to task completion by adding up the time users needed and dividing it by the total number of respondents.

User Error Rate

It goes without saying that it is super interesting to zoom in deeper ornto mistakes that users are making when performring the usability tests.  It gives you an idea of ​​the user-friendliness of the product and shows you what users encounter.

Apart from the total number of mistakes that respondents make, you can also compare which interfaces promote errors or provide more clarity (e.g. with a/b testing).

But when does the user make a mistake?

Before you can calculate the error rate, you want to be clear on when something is an error and when it is not. You can think of opening the wrong page, selecting the wrong product, entering incorrect information in the contact form, et cetera.

This depends on the type of product you develop. To ensure that the error rate does not paint a distorted picture, it is advisable to record this in advance in the usability test plan.

When you are clear when something is an error, you can calculate the error rate with the formula below. For this you sum up the total number of mistakes made. You divide this by multiplying the total number of errors possible and the total number of respondents.