Usability testing - what is it and why use it?

Usability or user-friendliness is the extent to which a user can use a product in his own way without obstacles, doubts, or questions. With a user-friendly product, the use of the product is smooth and the user does not experience frustration.
 

Definition of usability

The ISO 9241-11:2018 standard provides the following usability definition:

“Usability is the degree of effectiveness, efficiency, and satisfaction of a defined group of users to achieve a specific goal in a specific context.”

Usability consists of three parts:

• Effectiveness. The extent to which users can use the product to achieve their goal(s).
• Efficiency. The effort that users have to make to achieve a certain goal.
• Satisfaction. This refers to the thoughts of the user while using the product. This includes the user's opinion about the ease of use, the added value of the website or app, and its applicability.

In usability testing, users perform tasks on a website, app, or prototype while being filmed as they continuously think out loud. 

Because the user provides constant feedback, it becomes clear why certain functionalities do or do not work, which translates into a list of concrete action points to make the customer experience (even) better. 

PS: Usability testing is also known as user testing.

For every phase in the development process of a website or app, testing with real users is of great value. The five biggest benefits are:

• Discover what users encounter. Convenience serves people, now more than ever. When users want to order a product, but the ordering process does not work, they order it from the competitor with three clicks. That leaves the owner of the webshop with one question: why did the user abandon its shopping cart? Through user testing, you discover the frustrations of your user audience that would normally go unnoticed, providing you with the advantage of knowing who your user is, how they want to receive your product, and how you need to provide it.
• A better end product with prototype testing. Save time and money by conducting usability testing during the development process. Is it clear what they can do with the app or website? What do they encounter? What would they use it for? By collecting and processing this feedback during the development process, you build a product that better suits the end user.      
• Create empathy for the end user. It often occurs that a team that has been working on a project for a long time, find it increasingly difficult to empathize with the end user, because they know the product inside out as an expert and no longer a first time user. User testing forces you to see the product through the eyes of the end user, which gives you a lot of valuable information for the rest of the development process.      
• Valuable addition to quantitative data. Tools such as Google Analytics and Hotjar show what users do, but not why they do it.. A heatmap can show you that users are dropping out during the ordering process, but not why users are still considering. Usability testing removes that uncertainty and provides insight into how the ordering process can be improved.      
• A better product for everyone. Whether you have just launched your product, or have been in the market for some time, through testing you can find out to what extent the product meets users' expectations. Is something missing in the interface? Do we lose money by having inefficiently arranged the ordering process?     

Usability research can be performed in different ways. When someone talks about usability research, they almost always talk about performing usability or user tests. Did you know that usability research has been conducted since the 1980s? 

The research methods for usability research can be distinguished on two dimensions. The first dimension is the location of the usability study: online or in a usability lab. The second dimension concerns the way in which the research is conducted: with or without a moderator.

In a usability lab

By taking the tests in a usability lab, you have maximum control over the equipment that is used and you can quickly intervene when there are technical problems. The main drawback is that the user is not in their natural environment, which affects the answers and makes users less honest.

It also costs more time and money to recruit users and have them travel to the usability lab. User testing in a usability lab is almost always accompanied by a moderator who interacts with the tester / respondent during the test. 

Remote / Online

A remote user test means that the tester can participate in the usability research online. In practice, this often means that the user test takes place from home or at the office. The main advantage of this is that the user takes the test in their natural environment, which leads to fairer feedback and makes it easier to reach difficult respondent profiles.

Another added benefit of remote user testing? It is often many times cheaper than in a usability lab. Where a test in a lab costs on average between €5,000 and €10,000, this can be done from €650 via an online platform such as User Sense. A world of difference!

Curious about the possibilities? Request a demo with us! 

Request a free demo
 

Moderated

This usability research method is also known as moderated user testing. The moderator guides the test session from start to finish. The test can take place both face-to-face and online. When the test takes place online, the moderator uses an online tool to communicate with the users.

The biggest advantage is that a moderator can ask in-depth questions and thus gather more insights. On the other hand, an (inexperienced) moderator can influence the results of the research with leading questions.

If this happens, it will have a negative effect on the reliability of the results - something you logically want to avoid.

Moderated usability research is most suitable if:

• You want to test and validate  a concept or idea
• You  want to test a wireframe, mock-up or non-interactive prototype
• You want to make a day out of testing, with the entire (customer) team watching the execution of the user tests

Unmoderated

Of course, it doesn't make sense to have testers travel to a usability lab and then not talk to them. This also means that this variant of usability research almost always takes place online. This variant is therefore called remote unmoderated usability testing (URUT) .

The biggest advantage of this variant of usability research is that the tester cannot be influenced by a usability researcher during the execution, which means that the tester exhibits its most natural behavior during the user test. In our experience, this often leads to fairer feedback and greater willingness to test, because the tester can perform the usability test at a time of his/her choosing.

Unmoderated remote usability research is most appropriate if:

• You want to improve an existing app or website based on qualitative research (= user testing)
• You want to test a clickable prototype so changes can be made before the app is built
• You want to conduct international usability research

This variant of user testing is almost always carried out via a platform such as that of User Sense. Are you looking for a tool to perform user testing? Then we advise you to take a good look at how the quality of the testers is guaranteed and the size of the pool of the country in which you want to test. 

How much does usability research cost?

The costs for conducting usability research start at € 650 and can sometimes rise to € 10,000. The price of the research depends on the chosen research variant, the location of the research and whether respondent recruitment is necessary. It applies that conducting UX research via a platform such as User Sense is almost always cheaper than on location.

With User Sense you can  conduct your own research from € 65 per user test or from € 1395 if you let the User Sense researchers do this. 

User Sense can help you conduct usability research in the Netherlands , Belgium,  Germany, France and the United Kingdom . Curious what we can do for you? Then request a demo from us

Starting usability research? Great plan! If we can give you one tip: make sure you take enough time to set up the research. The more precisely you formulate the research questions and draw up the test plan, the more useful the results will be. Good luck!

Step 1: Data Deep Dive

Do you want to test an existing website? Then you get the most out of the usability research by including the data you have already collected. Google Analytics, Hotjar or Clarity are an excellent starting point for this.

Questions you can ask yourself are:

• Which pages have the highest bounce rate?  
• Which type of device has the lowest conversion rate?
• Where do the biggest conversion leaks seem to be?

Now that you have mapped out the bottlenecks of the website, you can organize the usability research in such a way that you find out what the biggest obstacles are.

No existing website, but want to test an app or prototype? Then there is a good chance that you have little quantitative data to draw from. That doesn't have to be a problem, as long as you think carefully about the research questions. 

Tip: No clear picture of what the target group looks like? Then use Google Analytics to better map out your target group. This helps you to test with respondents who most closely match your target audience.

Step 2: Formulate research questions

The next step is to formulate a clear main question and sub-questions. Drawing up clearly defined main and sub-questions helps you to get the most out of the usability research. 

This has a number of advantages:

• It helps you maintain focus
• You assure yourself that the research provides answers to the questions that have the most impact
• Main and sub questions are easy to communicate and help you get buy-in from stakeholders

Step 3: Choose the right usability research method

Now that it is clear what exactly you want to research, you can look at which methodology is best suited.

Do you want to test a concept or idea? Or get feedback on a mock-up or non-clickable prototype? Then you are often best off with moderated usability research.

Want to test a clickable prototype or an existing website or app? Then a moderator is often not needed and you can save a lot of time and money by performing unmoderated user testing.

Step 4: Determine the respondent profile and sample size

You prefer to conduct usability research with respondents who match the target group. It is therefore important to have a good idea of ​​what your target group looks like.

Example respondent profile:

• Respondents must be between 30 and 40 years old
• Respondents must have a higher than average income
• Respondents must have booked an intercontinental flight in the past year

Need help recruiting respondents? Take advantage of our respondent pool.

In general, we recommend that customers test with a minimum of five testers per device type. You can read more about respondent numbers for usability research in this article.

Step 5: Compose the usability test plan or script

Now that it is clear what you want to get from the user research and with whom you want to conduct the research, it is time to make the implementation concrete. To do this, you draw up a usability test plan, in which you record the tasks that testers must perform.

This way you ensure that testers do not stray too much during the research and that you receive feedback on aspects that are most important to you as a company.

Do you find it difficult to create a user test plan? In our whitepaper we help you on your way with a number of slides.

Step 6: Conduct usability research

It's finally time: you can start carrying out the usability research. In the case of moderated test sessions, this means that you can start recruiting respondents and then schedule the test sessions with them.  

Would you like to conduct unmoderated usability research yourself? Then you can use User Sense's self-service. After going live with the test, you can sit back and watch the test results roll in.

Step 7: Analyzing research results

Would you like to analyze the results of the usability test sessions yourself? Then make sure you take enough time for this step. A lot of time is spent reviewing the user tests, taking notes, and tracing patterns.

Because analyzing the results of usability research takes a lot of time, many companies choose to outsource the analysis. We also help companies with this.

The advantage of performing user tests via a tool is that you can also easily collect a number of quantitative usability metrics, such as:  

• System Usability Scale (SUS)
• Net Promoter Score (NPS)
• Single Ease Question (SEQ)
• Task-to-completion rate, time-on-task and error rate (more on that in this article)

Step 8: Create a UX research report

Perhaps the most satisfying step is to summarize the most important findings in a UX research report, so that you can answer the main and sub-questions, and it immediately becomes clear how you can improve the UX and conversion.  

Tip: Are you going to present the research results? Then make sure you generate a highlight video of the most interesting fragments from the test sessions. This immediately makes it clear why something needs to be improved!