Choose The Best Usability Test For Each Client

UX Design | 03th June 2017

The digital reality we currently live in, allows users to have a wider range of options to choose in a highly competitive market offering the same product or service, which can reached by a click. For these reasons, delivering the best user experience possible is a way to stand out in the market and to retain your clients.

For instance, when it comes to mobile apps, it’s common knowledge that a negative experience is enough to make an app never be used again. For example, only 16% of the people who were interviewed for this TechCrunch’s research claimed that they would use again an app with which they had a previous bad experience.

The usability is defined as “measure in which a product can be used by specific users in order to reach specific goals with effectiveness, efficiency and satisfaction, in a specific using context” (norm ISO 9241-11).

Apart from all these motives, there are two strong reasons as to why you should worry about usability, from the beginning  of your project:

  • Reduction of developing costs and time, due to predictability
  • An increase in attraction as well as retention of users and clients, due to a good experience.


How can I analyse a product’s/service’s usability?

Here at Hi INTERACTIVE, the most common ways of investigating a digital interface’s usability (app, website, system, etc.) are through a specialist’s analyses or interviewing the users.

The first one is a heuristic evaluation, and it consists of having one of our usability experts running an analysis on the interface to be developed, evaluating it according to some appropriate heuristics for each project.The second one consists on asking the real users to do certain tasks in specific using contexts.

Each one of these approaches has its advantages and disadvantages, given that the tests with users are the most complete ones, when it comes to results that end up being more aligned with user's real needs. However, they demand more work hours, which reflects on extended deadlines, and an increase in the project's investment.

What is heuristic evaluation?

Heuristic evaluation is a well recognised and proper practice which is capable of bringing value to your project in a short time frame (but without the depth and insights achieved through actual contact with the real users).

It is possible to design good interfaces based on the insights gained from heuristic evaluation. However, since the user experience is very extensive, the ideal scenario would be that each project had a specific time dedicated to research with real users, and relying on the tests made by a mixed group of users which represent the product's target audience. All of this should be developed under a usability specialist´s supervision, that will translate the users’ reactions, feelings, resourcefulness, difficulties, doubts and hesitations, in a list of problems to solve and improvement opportunities.

For best results, this work is done before the beginning of  IT Development, doing it so will allow all the insights to be identified before many hours of development, in a way that the changes detected could be applied without a significant impact on the project’s budget.


How should the tests be carried out?

Each test’s specific goal depends on the project and evolves managing:

users – project goals – using context

A question which usually pops up, concerns about the number of participants necessary to perform a test. According to  Jakob Nielsen, testing with 3 to 5 people (of every target audience´s profile) is the best cost-benefit ratio for a project.

Usability issues found x number of user tests


Where are these tests applied?

The tests can be made straight in the real user’s environment (real using context) - if it's allowed so - or at the Hi INTERACTIVE´s offices. Usually, when held in the client's offices, the users tend to mask their negative reactions, which ends up harming the test´s final score.


What sort of tests can be applied?

There are a series of approaches possible for the execution of the tests with the users:

  • Test with users pre-project
  • ​Tests with users pre-development
  • Tests with users pre-launching
  • Post launch A/B test

Each project will be best answered to, within its specificities, with one of these options, or with the combination of some of them.

1. Tests with users pre-project

This approach is appropriate for the already existing interfaces’ redesign. In the interview, we run a list through the users, with tasks to accomplish on the existing website. Afterwards, we keep it recorded and evaluate their behaviour: what was easy, the moments of struggle, what pleases them in the experience and what they would like to be different.

This interview is recorded and a deep analysis is made to translate its findings into points that require attention as well as project improvements.

The goal is to evaluate the design's preliminary effectiveness and also to know how the users think.

2. Tests with users pre-development 

In this case, the tests are made in a low-reliability prototype (wireframes) in which the proposed interface is presented to the users, so that they do a group navigation following a list of tasks.

During the execution of the tasks, we ask the user to speak out loud what he is thinking so to evaluate his behaviour: what was easy, struggling moments, what pleases him the most and what he would like to be different.

This interview is recorded, and a deep analysis is made to translate its findings into points that require attention as well as project improvements.

The primary goal is to evaluate the usability on a more concrete level of action, as well as the product´s aspects, assessing how the users perform in real tasks, and identifying potential specific usability problems. At this stage, the focus is more directed towards the user's behaviour, instead of the way he thinks.

3. Tests with users pre-launching  

These tests are executed in the final website, and we ask the users to perform a list of tasks, speaking out loud what he thinks so that we can evaluate his behaviour: what was easy, struggling moments, what pleases him the most and what he would like to be different.

This interview is recorded and a deep analysis is made to translate its findings into points that require attention as well as project improvements.
The primary goal is to certify the pre-launching product’s usability (in some cases, afterwards) checking how the product responds (or not) to the user experience goals. In this case, it is necessary to predict work hours for the execution of possible changes identified as needed.

4. Post-launch A/B test

An A/B test consists of presenting different versions of the same page, with little variations, and after a certain time or number of accesses, identify the best result version.

We use it in situations where we have doubts on which approach is more efficient on a page or when we intend to make a change but we’re not sure if it will have an adverse impact on the overall experience.

This test's primary goal is to obtain information from the website's real users and make well informed decisions before making the final change. Besides, it also allows you to understand how certain individual elements of an interface impact global experience.

Among others, the factors that can be altered in a page, after this test, are:

  • The order of a page's elements;
  • Call-to-actions;
  • Pictures;
  • Positioning of the menu and its behaviour.

Our main focus in this test is understanding the goals that you want to see accomplished with the change, and propose variables and solutions to be tested. For the majority of the projects, three variables (including the one being used at the moment) is a good number.


What are the advantages of investing in these tests?

The main advantage of testing with users is the possibility to argue and make decisions based on facts and not on each's taste. It also allows proving in advance, if what you are offering is really what your client is willing to use (and pay for), along with the opportunity to know your customers better, understand how they think, how they use the interface and what’s their relationship with technology.

This concentrated effort to understand their real needs has a very significant return on the investment, regarding higher engagement with your clients, as well as a lower development and maintenance cost, diminishing the changes identified solely through the customers’ complaints.

Studies show high financial return when the usability is used effectively. This way, investing in UX is not just something for “big companies” or for more “advanced” projects, but a plus in any sized project.


>> Read here for more on the affinity between a good user experience and customer loyalty