Provides a basic overview of and general information about usability.

• What Is Usability?
• Why Is Usability Important?
• What Is the Difference Between Usability Engineering and Usability Testing?
• What Steps Are Involved in Usability?
• What Is a Usability Test Like?
• Do I Need a Lab to Do Usability Testing?
• How Many Participants Are Needed for a Usability Test?
• How Much Does It Cost to Do Usability Testing?
• Why Should I Do Iterative Usability Testing?
• How Can I Show That Usability Engineering Saves Money?
• How Can I Encourage People in My Organization to Conduct Usability Engineering and Testing?

What Is Usability?

Usability is the measure of the quality of a user's experience when interacting with a product or system — whether a Web site, a software application, mobile technology, or any user-operated device.

Usability is a combination of factors that affect the user's experience with the product or system, including:

Ease of learning	How fast can a user who has never seen the user interface before learn it sufficiently well to accomplish basic tasks?
Efficiency of use	Once an experienced user has learned to use the system, how fast can he or she accomplish tasks?
Memorability	If a user has used the system before, can he or she remember enough to use it effectively the next time or does the user have to start over again learning everything?
Error frequency and severity	How often do users make errors while using the system, how serious are these errors, and how do users recover from these errors?
Subjective satisfaction	How much does the user like using the system?

 

• Links to Related Articles

Usability and the Web: An Overview, by George Murray and Tania Costanzo, at the National Library of Canada,
http://www.nlc-bnc.ca/9/1/p1-260-e.html

What Is Usability?, by Environmental Systems Research Institute, Inc.,
http://www.esri.com/software/usability/whatisusability.html

Why Is Usability Important?

Research by User Interface Engineering, Inc., shows that people cannot find the information they seek on Web sites about 60% of the time. This can lead to wasted time, reduced productivity, increased frustration, and loss of repeat visits and money.

Other sources report:

• "There are about 43 million Web sites, and no one knows which ones are usable. The best sites we've found are usable only 42 percent of the time, and none that we have studied are usable a majority of the time ...."

• Studies by Forrester Research estimate several costs of bad site design. The two most striking are:

• Losing approximately 50% of the potential sales from a site as people can't find what they need

• Losing repeat visits from 40% of the users who do not return to a site when their first visit resulted in a negative experience

• Site design guru Jakob Nielsen reports:

"Studies of user behavior on the Web find a low tolerance for difficult designs or slow sites. People don't want to wait. And they don't want to learn how to use a home page. There's no such thing as a training class or a manual for a Web site. People have to be able to grasp the functioning of the site immediately after scanning the home page — for a few seconds at most."

• Links to Related Articles

Mazed and Confused, Sari Kalin, CIO WebBusiness, http://www.cio.com/archive/webbusiness/040199_use.html

Failure of Corporate Websites, Jakob Nielsen, Alertbox, http://www.useit.com/alertbox/981018.html

Web-Site Usability: Usability On The Web Isn't A Luxury, Jakob Nielsen and Donald A. Norman, in InformationWeek, http://www.informationweek.com/773/web.htm

What Is the Difference Between Usability Engineering and Usability Testing?

Usability engineering is a methodical approach to producing a Web site or any user interface. It is a practical and systematic way to deliver a product that works for users. Usability engineering involves several methods, each applied at appropriate times, including gathering requirements, developing and testing prototypes, evaluating design alternatives, analyzing usability problems, proposing solutions, and testing a site (or other interface) with users.

Usability testing is part of the process of usability engineering. Usability testing includes a range of methods for having users try out a site (or other system). In a typical usability test, users perform a variety of tasks with a prototype (or other system) while observers record notes on what each user does and says. Typical tests are conducted with one user at a time or two users working together. Testing may include collecting data on the paths users take to do tasks, the errors they make, when and where they are confused or frustrated, how fast they do a task, whether they succeed in doing the task, and how satisfied they are with the experience. The goal of most usability testing is to uncover any problems that users may encounter so those problems can be fixed.

• Links to Related Articles

Usability Engineering for the Web, Keith Instone, World Wide Web Journal, http://www.w3j.com/5/s3.instone.html

Usability Glossary, Usability First,
http://www.usabilityfirst.com/glossary/glossary.taf

What Steps Are Involved in Usability?

• Getting Started — Planning the Web Site

The first step is to understand:

• why you are developing a site
• who should come to your site
• when and why those people might come

In answering these questions, you establish your objectives for the site. The specific objectives depend, of course, on your organization and your audience.

You should also think about usability objectives for the site. General usability objectives are that a site must be:

• easy to learn
• efficient to use
• easy to remember on subsequent visits
• satisfying, with a minimum number of errors as users go through the site

All the usability objectives are important for most sites, but you may emphasize different ones for different audiences and situations. For example, in a site that is aimed at members of the general public who may only visit once in a while, you should build a site where almost no learning needs to take place to use it efficiently.

See also Methods: Planning the Web Site

• Collecting Data from Users

Because the design is to be based on user needs, data must be collected about those needs and how well an existing Web site (if there is one) is meeting those needs. There are a variety of ways to collect that data, including feedback forms and system metrics (log data on an existing site), and usability testing of the existing site.

See also Methods: Collecting Data From Users

• Developing prototypes

It is easier for a user to react to an existing example than to theorize what would work best. Useful results can be obtained by building a prototype site, with a minimum of text content and no graphics, for a first round of usability testing. The prototype can then be used to elicit user comments and observe the prototype's ability to lead the users through the tasks they need to perform. It can be built on paper or with simple HTML.

See also Methods: Developing Prototypes

• Collecting, writing, or revising content

Based on what users need, you must put content into the site. As you consider information that you already have, think about how useful and understandable it is. Reading from a computer screen is slower than reading from paper. Most people want to quickly scan information and read only small sections. If the information you have is in long paragraphs, consider revising it. Break it into small chunks with many headings. Cut out unnecessary words. Use lists and tables so people can find information quickly. Follow these same guidelines when writing new information for the Web.

See also Methods: Collecting, Writing, and Revising Content

• Conducting usability tests

Usability testing is an iterative process. The goal of usability testing is to ascertain what will help users accomplish their tasks and what may impede them. Using the prototype as a starting point, the usability testers build a set of scenario tasks they will ask users to attempt. As detailed information about user success is gathered and reported, the prototype can be modified and additional aspects of that prototype tested.

Usability testing can be done inexpensively or more formally, depending on the size and budget of the site under development. As the testing team becomes more experienced, testing can be accomplished more quickly.

See also Methods: Conducting and Using Usability Tests

• Continuing to assess the site after it is up

When the site has been implemented, it is incumbent on the developers or the owners of the content to assess its performance by analyzing reports, usage logs, and other data sources for the site and by continuing to gather user feedback on usability.

See also Server Log Analysis

Log Analysis links at Usable Web
http://www.usableweb.com/topics/000649-0-0.html

• For more information

See also Methods for Designing Usable Web Sites

What Is a Usability Test Like?

The focus of a usability test is the user's experience with a site. The site may exist only as a paper prototype, or it may be a real working prototype or a site that has already been launched. The earlier you start to have users try out the site, the faster and easier it will be to develop the site you want.

During a usability test, specialists working with the designers and developers of the site watch users working through tasks with the site and gather other feedback. The purpose is always to see what is working well and what is not working well — with the goal of improving the site. Usability specialists manage the test, work directly with the users, and take notes; designers, developers, and others also observe — usually from an adjacent room or from a live video and audio stream — and should also take notes.

Usability specialists gather all the notes, consolidate and analyze them, and together with the designers and developers consider the problems they saw and how to solve them. The result of usability testing is a set of recommendations for improving the site.

Do I Need a Lab to Do Usability Testing?

No. You can do usability testing in either a formal or informal setting. In any type of setting, your methodology can also range from formal to informal.

You can do effective usability testing in any of these settings:

• a fixed laboratory having two or three connected rooms outfitted with audio-visual equipment
• a conference room, or the user's home or work space, with portable recording equipment
• a conference room, or the user's home or work space, with no recording equipment, as long as someone is observing the user and taking notes
• remotely, with the user in a different location, as long as the observer can monitor what the user is doing, listen to the user thinking aloud, and interact with the user by computer or telephone during the session

For more on audio-visual equipment for recording usability test sessions, see the lists of equipment for both fixed and portable labs in Usability Labs.

• Links to Related Articles

Usability Labs: Our Take, in User Interface Engineering,
http://world.std.com/~uieweb/labs.htm

How Many Participants Are Needed for a Usability Test?

It depends. A typical range is from five to 12 users in each test. If each user works with you for an hour, that means one or two days of testing.

You might need only three people to help you find serious problems, if you:

• are doing paper prototypes or are in early development
• plan several rounds of testing throughout development
• have a fairly homogenous user population

If you have different potential user groups (for example, physicians, patients, researchers), try to include representatives of all these groups. If you are likely to have users with a range of Web or computer experience, try to include both less experienced and more experienced users. Those considerations may push the number of people you need from three to six or nine or 12.

If you want to conduct formal quantitative testing on your products or systems, you'll need more people to derive statistical results. For diagnostic usability testing, five users are usually enough to uncover the major problems in a product.

If you do iterative (repeated) usability testing over the course of developing the Web site, many users will participate in testing one or another version of the emerging site. Thus, while you may have fewer than 10 participants in each usability test, you may have 15 to 30 people who have tested some version of the site before it is launched.

• For more information on:

Different goals for usability testing (diagnostic, comparative, verification), see How formal or informal should usability testing be?

Deciding which type of test to conduct paper prototyping, see Developing Prototypes

Needing only five people to find major problems, see Jakob Nielsen, Why You Only Need To Test With 5 Users, Alertbox, March 19, 2000, http://www.useit.com/alertbox/20000319.html

How Much Does It Cost to Do Usability Testing?

Cost depends, of course, on the size of the site, how much you need to test, how many different types of users you anticipate having, and how formal you want the testing to be.

Having a standard process and reusable materials makes usability testing faster and less expensive. If you or your recruiting firm develop a database of users, recruiting becomes less time consuming and, therefore, cheaper.

Consider these elements in budgeting for usability testing:

• time to plan: identify issues to focus on in testing, identify types of users to involve in testing, write a screening questionnaire to recruit users, write scenarios for users to follow
  
  
• cost of recruiting: time of in-house person or payment to a recruiting firm (often a good option)
  
  
• time of usability specialist to become familiar with the site and of team to do a dry run to see how scenarios work with the site
  
  
• cost of renting laboratory space or a portable lab or other videotaping equipment if you do not have free access and want to record on videotape
  
  
• time of team to observe users (conduct the test)
  
  
• cost of paying participants or gifts for participants
  
  
• time to consider what the team saw and heard, identify problems, recommend solutions to those problems
  
  
• time to discuss changes with developers, write up memo of findings and recommendations

Remember to budget for more than one usability test. Building usability into a Web site (or any product) is an iterative process. You will find it more valuable to use your budget to do a few small tests throughout development than to do just one large test at the end. Each small test might include anywhere from three to 10 users.

If you are not sure why to do several small tests, see Why Should I Do Iterative Usability Testing?

If you need to convince people that usability testing is worth the cost, see:

The evidence we cite in How Can I Encourage People in My Organization...?

How to calculate the cost-benefit of usability in How Can I Show That Usability Engineering Saves Money?

Why Should I Do Iterative Usability Testing?

A few small tests are more valuable than one large test at the end.

• The sooner you find problems, the less expensive it is to fix them.
  
  
• Finding and fixing problems early means less rework. That not only saves money, it reduces designers' and developers' frustration.
  
  
• You can test branding (Do people realize whose site it is?), navigation (Can people find what they need?), and organization of the home page before you have developed all the content or coded the entire site.
  
  
• You can test many design issues with paper prototypes and then test again when you have a working site. See Developing Prototypes.
  
  
• You can test one part of the content and learn valuable lessons to apply to other parts that aren't yet developed.
  
  
• Changes are more likely to get made early in the development process. Cost, time, and human reluctance usually lead to ignoring problems that are found if you test only at the end.
  
  
• Testing once isn't enough to be sure you have a usable site. Use iterative testing to see whether the way you fixed an earlier problem really works for users.
  
  
• You can use iterative usability testing to show how the benefits of usability engineering greatly outweigh the costs. See How Can I Show That Usability Engineering Saves Money?

How Can I Show That Usability Engineering Saves Money?

You can use usability testing to show that the benefits of usability engineering outweigh the costs.

This method was first published by Clare-Marie Karat of IBM who used it to show a 100-fold return on investment for a particular software product. In that case, spending $60,000 on usability engineering throughout development resulted in savings of $6,000,000 in the first year alone.

The results from this technique are especially convincing if the same organization pays both the development costs of the Web site and the salaries of the people who use the site. But it should also be convincing to organizations that really care about how problems on their site cost their external users time, money, and frustration.

The types of problems that you might find costing time (and therefore money) are misleading navigational cues, poorly designed pathways, pages that are so dense they take a long time to use, etc.

Here is how you can use usability testing to show how benefits outweigh costs:

1. Do a usability test on an early version of the Web site (or other product) — this could be the old site or one done without involvement of usability specialists.

• Use actual users doing relevant tasks.
• Measure time to complete tasks.
  

2. Identify and fix problems. (Improve the entire site not just the test tasks even though you will be using the test tasks to show the benefits.)

3. Do a usability test on the new version of the site.

• Have users who match the demographics of the first set of users do the same tasks you used in the first test.
• Measure time to complete the same tasks.

4. Calculate the improvement in average time to complete each task.

• You can do the next steps for each task separately, for just one major task, or for all the tasks together.

5. Multiply the time saved by the number of people who are likely to do that task in a given time period (say, each day).

• If users are likely to do a task several times a day, you can also multiply by that number.
• If you have noted the time saved in seconds or minutes, convert it to hours because you will want to work in hours in the next step.

6. Identify the average hourly salary of the users who do that task.

7. Convert time to dollars by multiplying time saved (in hours) by users' salary (per hour).

8. Find the one-year savings by multiplying your previous figure by the number of days in the year that users are likely to do the task.

• If this is a work task, use the number of days in the organization's working year.
• You now have the total annual savings of your usability changes — all due to time saved by fixing the product so users can do tasks more quickly.
  

9. Compare the amount saved to the cost of usability activities.

• For More Information:

Karat, C.-M. Business case approach to cost justification. In Cost-Justifying Usability, Edited by R. G. Bias and D. J. Mayhew, Boston: Academic Press, 1994, 45-70.

How Can I Encourage People in My Organization to Conduct Usability Engineering and Testing?

If you're trying to persuade the people in your organization to use usability engineering to design your Web site:

• Quote these statistics:

• Research by User Interface Engineering, Inc., shows that people cannot find the information they seek on a Web site about 60% of the time.


• According to Elizabeth Millard, "The best sites we've found are usable only 42 percent of the time, and none that we have studied are usable a majority of the time...."

• Studies by Forrester Research estimate that approximately 50% of potential sales are lost from a site as users can't find data and that 40% of users do not return to a site when their first visit is a negative experience.

• A study by Zona Research found that 62% of Web shoppers have given up looking for the item they wanted to buy online.


• Let them observe a usability test or show them video footage. Observation of user behavior is very persuasive. The user perspective is just about impossible for Web site production teams and content developers to see without talking to or observing actual users.
  
  
• Remind them that usability testing doesn't have to be expensive. Big problems in the design are obvious after just a few user tests. Jakob Neilsen says, "You sit somebody down in front of the screen and see immediately if they click on the right button or the wrong one. As soon as you see three people make the same mistake, you're better off just fixing it."

• Links to Related Sites

Failure of Corporate Websites, Jakob Nielsen, Alertbox, http://www.useit.com/alertbox/981018.html

Usability as Barrier to Entry, Jakob Nielsen, Alertbox, http://www.useit.com/alertbox/991128.html

• Moving Forms to the Web - Thursday-Friday, October 14-15, 2004

• Goal Oriented Planning and Testing October 21, 2004

• New Research-Based Guidelines Sorting Tool: Sort Guidelines by Importance, Strength of Evidence, and Other Options

• What's New Archive

Department of Health
and Human Services

FirstGov.gov