What is Usability Testing

What is Usability Testing - Part 2


Planning a Test

The first thing to know about planning a usability test is that every test is different in scope, and results will vary a lot depending on the purpose and context of the test. Testing a single new feature will look very different from testing several key scenarios in a new Application.

What Are You Going to Test?

Next, you need to decide what you’re going to test. The best way to do this is to meet with the design and development team and choose features that are new, frequently used, or considered troublesome or especially important. After choosing these features, prioritize them and write task scenarios based on them. A task scenario is a story that represents typical user activities and focuses on a single feature or group of related features. Scenarios should be:

- Short - Time is precious during usability testing, so you don’t want to spend too much time on reading or explaining scenarios.

- Specific - The wording of the scenario should be unambiguous and have a specific end goal.

- Realistic - The scenario should be typical of the activities that an average user will do on the Application.

- In the user’s language and related to the user’s context - The scenario should explain the task the same way that users would. This emphasizes the importance of the pre-session discussion, which gives you the opportunity to understand the participant’s relationship with the Application.

Here’s an example scenario for an Application that sells images:

Ex: You’re looking for an image that you can use on your company’s support Application. Find an appropriate image and add it to your basket. Be sure to let me know when you’re done.

Who is going to evaluate the Application?

Who you choose to evaluate the Application will have a massive effect on the outcome of the research. It’s very important to develop a thoughtful screener for recruiting your participants.

Imagine that you’re creating an Application that sells images. Your customers are people who want to buy images—a huge group of people. Narrow your focus to a short and concise user profile, a picture of your ideal test participants. This profile should be based on your primary user (customer) segment and contain characteristics that those users share.

In this scenario, our participants are graphic designers or other people who use graphic design software and purchase images online. Create and order a list of these users’ characteristics. While you’re creating the user profile, you may realize that you have two or more equally important subgroups—people who buy images for business use and people who buy images for home use. This is fine as long as you can justify the relevance of each subgroup to the features that you’ll be testing.

- Test with a reasonable number of participants —The best results come from testing no more than 5 users and running as many small tests as you can afford.

The most striking truth of the curve is that zero users give zero insights.

As soon as you collect data from a single test user, your insights shoot up and you have already learned almost a third of all there is to know about the usability of the design. The difference between zero and even a little bit of data is astounding.

When you test the second user, you will discover that this person does some of the same things as the first user, so there is some overlap in what you learn. People are definitely different, so there will also be something new that the second user does that you did not observe with the first user. So the second user adds some amount of new insight, but not nearly as much as the first user did.

The third user will do many things that you already observed with the first user or with the second user and even some things that you have already seen twice. Plus, of course, the third user will generate a small amount of new data, even if not as much as the first and the second user did.

As you add more and more users, you learn less and less because you will keep seeing the same things again and again. There is no real need to keep observing the same thing multiple times, and you will be very motivated to go back to the drawing board and redesign the Application to eliminate the usability problems.

After the fifth user, you are wasting your time by observing the same findings repeatedly but not learning much new.

You need to test additional users when the Application has several highly distinct groups of users

Article is prepared by:
Anushka Wickramaratne
Senior QA Engineer

What is Usability Testing

What is Usability Testing - Part 1

Usability testing is a technique for ensuring that the intended users of a system can carry out the intended tasks efficiently, effectively and satisfactorily. In other words, Usability testing should be an iterative practice, completed several times during the design and development life-cycle. The end result is an improved product and a better understanding of the users that we’re designing it for.

Main steps in the usability testing process are Planning, Gathering Data and Reporting results.

There are two scenarios for usability testing:

1. If you are a software product vendor, testing real users of your product means you are evaluating for design. Based on how you have designed the application, can users complete the tasks they need to do? Testing real users doing real tasks can also point out if the UI guidelines you are following are working within the context of your product, and when consistency helps or hinders the users’ ability to do their work.

2. If you are a software product purchaser, you can do usability testing to evaluate a product for purchase. For example, your company might consider buying a product for their twenty thousand employees. Before the company spends its money, it wants to make sure that the product in question will really help employees do their jobs better. Usability testing can also be useful to see if the proposed application follows published UI style guidelines (internal or external). It’s best to use UI guidelines as an auxiliary, rather than primary, source of information for making purchase decisions.

This Guide is discussing on the first scenario ‘evaluating for design’.

When Usability Testing is appropriate?

Usability testing is carried out pre-release so that any significant issues identified can be addressed. It can be carried out at various stages of the design process as well. In the early stages, however, techniques such as walkthroughs are often more appropriate.

Article is prepared by:

Anushka Wickramaratne

Senior QA Engineer

What is a Test Case

In simple terms, test cases are how the testers validate that software meets customer requirements. It’s a tool which helps testers to test a product.

Test case is a set of conditions organized in a manner which determines a requirement is met in the product by testers. Then testers can decide whether the test is passed or failed based on the output of the test case. If the output is passed the conclusion is that the tested requirement is functioning correctly. If test case failed then that requirement needs a code change to fix the issue. When a test case is failed, testers called it as bug/issue* of the system.

The main inputs for test cases are requirements of the system. This can in format of a requirement document which represents using deferent techniques such as use cases. Then it’s testers’ responsibility to design test cases to cover all requirements. There are tests to cover the positive test and the negative test for a requirement as the minimum.

There are several information required for a complete test case and those are,

Test case ID – to identify a test case, it should have a unique number.

Test case description – This includes the description of the feature or the requirement that’s going to test by this test case.

Steps – steps should be included to guide the tester to execute the desired test case

Input data – If the test case requires the specific data, those can be included here. When designing the test case, it’s acceptable to include the test data among the steps.

Reference – this will link the test case to the requirement which has documented in requirement document.

Expected result – this the expected output of the test case and how the feature/requirement should function in the product.

Testing status– this will contain information whether the test case is passed or failed after execution of the test.

Author – This will contain information who has designed the test cases.

Date – this will hold the information when the test cases are designed.

Change history- this will give an idea what has been changing in the test cases after creation of it.

Author, Date and change history can be single entry for a set of test cases.

*Bug/Issue will be discussed in detail in another post.

Vocabulary...

Followings are some of the words that will be useful for the next topic.

Risk is the potential loss to an organization, as for example, the risk resulting from the misuse of its computer. This may involve unauthorized disclosure, unauthorized modification, and/or loss of information resources, as well as the authorized but incorrect use of a computer. Risk can be measured by performing risk analysis.

Risk Analysis is an analysis of an organization’s information resources, its existing controls, and its remaining organization and computer system vulnerabilities. It combines the loss potential for each resource or combination of resources with an estimated rate of occurrence to establish a potential level of damage in dollars or other assets.

A Threat is something capable of exploiting vulnerability in the security of a computer system or application. Threats include both hazards and events that can trigger flaws.

Vulnerability is a design, implementation, or operations flaw that may be exploited by a threat; the flaw causes the computer system or application to operate in a fashion different from its published specifications and to result in destruction or misuse of equipment or data.

Control is anything that tends to cause the reduction of risk. Control can accomplish this by reducing harmful effects or by reducing the frequency of occurrence.

Reference: CSTE CBOk v 6.2