Choose The Best Usability Test For Each Customer

UX Design | 03th June 2017

The digital reality in which we live in is something that provides the users with an ever-growing number of options to choose from, and foster a wide number of competitors who offer the same product or service just a click away. For these reasons, provide the best user experience is a market differential and a way to nurture loyalty among your customers.

When it comes to mobile apps, for instance, it is acknowledged that a negative experience is what it takes for them to never be used again. For instance, solely 16% of the interviewees, of this TechCrunch research, have said that they go back to an app after having endured a poor experience with it.

Usability is defined by the “extent to which a product may be used by specific users to reach specific targets with effectiveness, efficiency and satisfaction within a context of a specific use” (standard ISO 9241-11).

In addition to all these, there are two strong reasons that should make you take usability into account, right from the project’s beginning:

  • Cost reduction and development time, due to predictability
  • ​An increase in user and customer attraction and retention, due to a good experience


How can one analyze the usability of a product/service?

Here at Hi INTERACTIVE, the most common methods to assess the usability of a digital interface (application, site, system, app, etc.) are the expert analysis and the user interview.

The first is the Heuristic Analysis, where one of our usability experts conducts an analysis of the interface to be developed, assessing it based on the presence or absence of some heuristics, which are regarded as appropriate for each project. The second is to request real users to conduct certain tasks in contexts of specific use.

Each of these approaches has their pros and cons, with user testing being the option that provides the most comprehensive results, attuned to the real user needs. Nonetheless, this requires a higher workload, which is reflected in a slightly longer deadline. ​


What is a Heuristic Analysis?

The Heuristic Analysis is an acknowledged and thorough practice which adds a great value to the project in a short period of time (although lacking the depth and the insights that are gathered when in touch with real users).

It is possible to outline good usability interfaces based on the outcomes of a Heuristic Analysis. However, since the user experience is quite encompassing, the ideal would be that each project had its own period devoted to research with real users, whilst having tests conducted with a mixed user group – which depict the product’s real target audience. All this should be developed under the supervision of a usability expert, which will translate the reactions, feelings, resourcefulness, struggles, conveniences and questions of the users, gathered in a list made of issues to be solved and opportunities for improvement.

In an even more positive scenario, this work is carried before the launch of the IT Development. This makes it possible to identify all insights before one devotes tons of hours to development purposes, hence allowing the execution of the identified changes without causing a major impact on the project’s budget.


How should one carry out the tests?

The specific goal of each test depends on components which need to be individually analyzed, for each project, and that should create a balance between:

users – goals – context of use

A common question, which ends up emerging, is related to the number of participants needed to perform a test. According to Jacob Nielsen, testing with 3 to 5 individuals (representative of each profile on the target audience) is the best cost-benefit ratio when the project does not have a budget to accommodate more comprehensive tests. Having in mind this indication, we usually suggest the participation of around 10 participants for each test, to make sure that at least 3 different participant profiles are analyzed.

Usability problems found x number of user tests
Why you only need to test with 5 users


When are the tests deployed?

The tests can be directly performed in the user’s real environment (real context of use) – if they allow so – or at the headquarters of Hi INTERACTIVE. Usually, when performed in the company’s office, users have the tendency to mask negative reactions, something that ends up impairing the test’s final result.


What sort of tests can be applied?

There are several sorts of approaches to conduct user tests:

  • Pre-project user testing
  • Pre-development user testing
  • Pre-launch tests user testing
  • Post-launch A/B testing  

Get fresh content about UX every month!Each project will be better addressed, within their own technicalities, if one of these options is met or if a combination of some of them is used.

1. Pre-project user testing

This approach is adequate for redesigning purposes on existing interfaces. During the interview, we hand out a list to the users, filled with tasks that they have to conduct in the existing website. Then, we track and assess their behavior: what was easy, the moments when they struggle, what pleases them the most in the experience, and what they would like to be different.

This interview is recorded and an in-depth analysis is conducted, which is translated into the project’s points of attention and improvements.

The goal is to assess the effectiveness of the preliminary design and be acquainted with the way the users think.

2. Pre-development user testing

In this case, the tests are carried out in low-fidelity prototypes (wireframes) in which the proposed interface is presented to the users, so that they navigate it alongside a task list.

As tasks are performed, we ask the users to say out loud what they are thinking and we assess their behavior: what was easy, the moments when struggles were felt, what pleases them the most in the experience and what they would like to be different.

This interview is recorded and an in-depth analysis of the user behavior is conducted after the tests.

The goal is to assess the usability on a more specific level related to the actions and aspects of the product, assessing how the user perform real tasks, and identifying possible issues specifically related to usability. In this stage, the focus is more concerned with the user’s behavior than with the way they actually think.

3. Pre-launch tests user testing

The tests are performed on the final website and we request the users to complete a task list. As tasks are performed, we ask the users to say out loud what they are thinking and we assess their behavior: what was easy, the moments when struggles were felt, what pleases them the most in the experience and what they would like to be different.

This interview is recorded and an in-depth analysis of the user behavior is conducted during the test.

The goal is to certify the usability of the pre-launch product (sometimes its post-launch as well), assessing if the product meets (or not) the user’s usability goals and experience. In this case, it is necessary to foresee the workload which may need to perform certain identified changes, as necessary.

4. Post-launch A/B testing

An A/B testing consists of presenting different versions of a page with slight variations and, after a certain period of time or a number of actions, of quantifying what version had the best result overall.

We use it in situations where there are some questions about what the most efficient approach for a specific page is, or when one wants to carry out a specific change, but there are questions if this will have any sort of negative impact on the final experience.

The goal of this test is to gather information from real users of the website and make well-reasoned decisions before we carry out the definitive change. Furthermore, it also makes it possible to understand how certain elements of an interface have an impact on the global experience.  

Among many others, the elements that may be changed in a page, after this test is performed, are:

  • The order of elements in a specific page;
  • Call-to-actions;
  • Images;
  • The positioning of the menu and its behavior.

Therefore, our focus with this test is to perceive the goals that one wants to reach with such change, and suggest variables and solutions to be tested. For most projects, 3 variables (including the one currently used) is a good number.  


What sort of benefits can I have if I decide to invest in these tests?

The number one benefit of these user tests is the chance to argue and make decisions based on facts and not on the opinion of every stakeholder. Furthermore, it allows ascertaining beforehand if what one is offering is actually what the customer is willing to use (and pay for), along with the opportunity to have a better knowledge of your customers, to know how they think, to how they use the interface and their relationship with the technology.

This concentrated effort to understand your real needs usually has a quite singular return on investment, since your customers will be more engaged, as well as a lower development and maintenance cost, reducing the changes identified only through customer complaints.

Studies prove the high financial return when one effectively relies on usability. Therefore, investing in UX is not only for “major companies” or more “advanced” projects, rather a proper advantage for any project regardless of its size.


>> Read more about the relationship between a good user experience and loyalty.

Looking for more information about UX?
Download this free eBook!
UX eBook