Supp
Notes: HEURISTIC EVALUATION
Kelly Booth
| |
The material here was mostly prepared by John Crawford of Interface Intelligence, mostly compiled from material available on the Web (sources are cited in each section). A heuristic evaluation is a systematic inspection (or "evaluation") of a user interface based on a set of usability principles (or "heuristics"). This type of usability test is valuable and cost effective because it can be performed early in the development cycle and can be executed without users. The main goal of heuristic evaluation is to find usability problems as early as possible so they can be attended to as part of an iterative design process. By defining and solving usability problems early on, this type of assessment ensures that these problems find solutions early on and save time and money. However, a heuristic evaluation can take place during almost any cycle in the design and development phase — from paper prototypes to a high-fidelity prototype. While there are many types of usability
evaluations — such as cognitive walkthroughs, usability lab testing and
contextual enquiry — heuristic evaluation is one of most cost-effective
assessments with a high return on investment (a 5 to 1 return according
to Nielsen, who evaluated multiple heuristic tests in 1994).
To ensure a thorough evaluation, it is important to have multiple evaluators. The numbers of evaluators is chosen based on the scope of the project and the budget. For a medium size project, for example, we recommend between 3 and 5, which is consistent with the findings of Nielsen, who conducted studies that indicate that with more than about 5 evaluators few new problems are discovered. Evaluators are traditionally usability or user interface experts. Based on the complexity and domain knowledge required for the project, domain experts may be used. For optimal results, evaluators should not be close to the development of the product. Generally, this type of evaluation lends itself to using an outside source. Once the evaluators are determined, the set of heuristics to be used should be outlined. There are different sets of heuristics available. Though evaluators may use a proprietary set of principles, evaluators more commonly use an industry standard set. It is important to limit the set of heuristics to a manageable number — generally, between 10 and 20. Much success has been achieved performing evaluations using a set of 10 heuristics originally created by Jakob Nielsen and then modified for the web by Keith Instone. You will notice that the list in Appendix 1 is somewhat different from the nine given in the text, which are Neilsen's original set. He later added "Help and Documentation" and reorganized his list based on a factor analysis of 249 usability problems that were found in his studies. Once the group of evaluators agrees upon the set of heuristics, they begin the evaluation. Each evaluator performs the inspection in isolation. During the inspection, each evaluator goes through the user interface several times, examining each screen and dialogue and comparing these elements to the list of heuristics. Non-compliance to the heuristics is rated by the "severity" and "extent" of non-compliance based on the following scale. Severity scale: 3 = Major usability problem Extent scale: 3 = Widespread After the evaluators have completed their independent evaluations, the entire group meets to compare and aggregate the results. The evaluators then categorize the problems based on consensus ratings of severity and extent, and generate a report. The results of a heuristic evaluation are a list of usability problems that reference the heuristics assessed and provide rationale. The problems are recorded in as much detail as possible, and each is noted separately. The results may highlight very simple and obvious fixes that have been overlooked, or may point to significant issues to be addressed by the product design team. Once the report is complete, all of the evaluators involved present the results to the client. The results of the evaluation do not include a redesign of your product; however, during the presentation of the results, the evaluators will likely offer possible solution ideas for certain problems, which can be incorporated into your redesign. The most important process in the
heuristic evaluation is what you do after you have receive the results.
Once you receive the results and meet with your consultants, work on a
plan for fixing the problems and improving your design.
[Based on the standard list developed by Jakob Nielsen. Additional web-oriented discussion (indented) is by Keith Instone, from http://webreview.com/97/10/10/usability/sidebar.html, however this link is currently broken. You can still see the original on the Way-Back Machine, however.] 1. visibility of system status The system should always keep users
informed about what is going on, through appropriate feedback within
reasonable time. Make sure each page is branded and that you indicate which section it belongs to. Links to other pages should be clearly marked. Since users could be jumping to any part of your site from somewhere else, you need to include this status on every page. 2. match between system and the real world The system should speak the users'
language, with words, phrases and concepts familiar to the user, rather
than system-oriented terms. Follow real-world conventions, making
information appear in a natural and logical order. 3. user control and freedom Users often choose system functions by
mistake and will need a clearly marked "emergency exit" to leave the
unwanted state without having to go through an extended dialogue.
Support undo and redo. Be careful when forcing users into certain fonts, colors, screen widths or browser versions. And watch out for some of those "advanced technologies": usually user control is not added until the technology has matured. One example is animated GIFs. Until browsers let users stop and restart the animations, they can do more harm than good. 4. consistency and standards Users should not have to wonder whether
different words, situations, or actions mean the same thing. Follow
platform conventions. "Platform conventions" on the Web means realizing your site is not an island. Users will be jumping onto (and off of) your site from others, so you need to fit in with the rest of the Web to some degree. Custom link colors is just one example where it may work well for your site but since it could conflict with the rest of the Web, it may make your site hard to use. And "standards" on the Web means following HTML and other specifications. Deviations form the standards will be opportunities for unusable features to creep into your site. 5. help users recognize, diagnose, and recover from errors Error messages should be expressed in plain
language (no codes), precisely indicate the problem, and constructively
suggest a solution. For example, if a user's search yields no hits, do not just tell him to broaden his search. Provide him with a link that will broaden his search for him. 6. error prevention Even better than good error messages is a
careful design which prevents a problem from occurring in the first
place. 7. recognition rather than recall Make objects, actions, and options visible.
The user should not have to remember information from one part of the
dialogue to another. Instructions for use of the system should be
visible or easily retrievable whenever appropriate. Certainly the most invisible objects created on the Web are server-side image maps. Client-side image maps are a lot better, but it still takes very well-crafted images to help users recognize them as links. Good labels and descriptive links are also crucial for recognition. 8. flexibility and efficiency of use Accelerators — unseen by the novice user —
may often speed up the interaction for the expert user such that the
system can cater to both inexperienced and experienced users. Allow
users to tailor frequent actions. Make pages at your your site easy to bookmark. If a user is only interested in one corner of your site, make it easy for him to get there. Better that than have him get frustrated trying to get from your home page to what he is looking for. Do not use frames in a way that prevent users from bookmarking effectively. Support bookmarking by not generating temporary URLs that have a short lifespan. If every week you come out with a new feature article for your site, make sure your URL lives on, even after the content is taken down. Web Review uses long-term locations by putting date information into the URLs. Or, you could re-use your URLs for the newer content. Consider using GET instead of POST on your forms. GET attaches the paramters to the URL, so users can bookmark the results of a search. When they come back, they get their query re-evaluated without having to type anything in again. All of these rules for "design to be bookmarked" also help you design to be linked to. If the contents of your site can easily be linked to, others can create specialized views of your site for specific users and tasks. Amazon.com's associates program is just one example of the value of being easy to link to. 9. aesthetic and minimalist design Dialogues should not contain information
which is irrelevant or rarely needed. Every extra unit of information
in a dialogue competes with the relevant units of information and
diminishes their relative visibility. The best way to help make sure you are not providing too much (or too little) information at once is to use progressive levels of detail. Put the more general information higher up in your hierarchy and let users drill down deeper if they want the details. Likewise, make sure there is a way to go "up" to get the bigger picture, in case users jump into the middle of your site. Make sure your content is written for the Web and not just a repackaged brochure. Break information into chunks and use links to connect the relevant chunks so that you can support different uses of your content. 10. help and documentation Even though it is better if the system can
be used without documentation, it may be necessary to provide help and
documentation. Any such information should be easy to search, focused
on the user's task, list concrete steps to be carried out, and not be
too large. For the Web, the key is to not just slap up some help pages, but to integrate the documentation into your site. There should be links from your main sections into specific help and vice versa. Help could even be fully integrated into each page so that users never feel like assistance is too far away.
By Bruce Tognazzini, from http://www.asktog.com/basics/firstPrinciples.html The following principles are fundamental to the design and implementation of effective interfaces, whether for traditional GUI environments or the web. Of late, many web applications have reflected a lack of understanding of many of these principles of design, to their great detriment. Because an application or service appears on the web, the principles do not change. If anything, applying these principles become even more important. Effective interfaces are visually apparent and forgiving, instilling in their users a sense of control. Users quickly see the breadth of their options, grasp how to achieve their goals, and do their work. Effective interfaces do not concern the user with the inner workings of the system. Work is carefully and continuously saved, with full option for the user to undo any activity at any time. Effective applications and services perform a maximum of work, while requiring a minimum of information from users.
Applications should attempt to anticipate the user’s wants and needs. Do not expect users to search for or gather information or evoke necessary tools. Bring to the user all the information and tools needed for each step of the process.
Give users some breathing room. Users learn quickly and gain a fast sense of mastery when they are placed "in charge." Paradoxically, however, people do not feel free in the absence of all boundaries (Yallum, 1980). A little child will cry equally when held too tight or left to wander in a large and empty warehouse. Adults, too, feel most comfortable in an environment that is neither confining nor infinite, an environment explorable, but not hazardous.
No autonomy can exist in the absence of control, and control cannot be exerted in the absence of sufficient information. Status mechanisms are vital to supplying the information necessary for workers to respond appropriately to changing conditions. As a simple example, workers, failing status information, will tend to maintain heightened pressure on themselves during slow periods, until the moment the work actually runs out. This will stress and fatigue them unnecessarily, so that when the next rush occurs, they may be lacking the physical and mental reserves to handle it.
Users should not have to seek out status information. Rather, they should be able to glance at their work environment and be able to gather at least a first approximation of state and workload. Status information can be quite subtle: the inbox icon could be switched to show an empty, somewhat full, or stuffed state. This, however, should not be overdone. The Macintosh, for years, showed an icon of a trashcan of imminent danger of explosion if a single document was placed therein. Users quickly formed the habit of emptying the trashcan as soon as the first document hit. This not only turned a single-step operation into a two-step operation (drag to the trash, then empty the trash), it negated the entire power of the trashcan, namely, undo. As another positive example, a search field icon can change color and appearance to indicate that the search is in progress or has been completed with too many matches, too few matches, or just enough. (Like any element of the interface, just color is not enough; 10% of males show some indication of color blindness. Even a higher percentage may have temporary alterations in perception of blue under varying conditions.)
The cones in the eye are the source of color vision. We have cones separately sensitive to red, green, and blue. If the red ones are not functioning that is called protanopia. If the green are not functioning, that is called deuteranopia. Absence of blue, extremely rare, is called tritanopia. Protonopia and deuteranopia are the most popular forms of color blindness, collectively called red/green blindness. (There are, in fact, significant differences in their effects, but those differences have no real effect on design.) While tritanopia is far more rare, it nonetheless rules out dependence on yellow-blue differentiation without secondary cues. Secondary cues can consist of anything from the subtlety of gray scale differentiation to having a different graphic or different text label associated with each color presented. The following principles, taken together, offer the designer tremendous latitude in the evolution of a product without seriously disrupting those areas of consistency most important to the user:
"Invisible structures" refers to
such invisible objects as Microsoft Word's clever little right border
that has all kinds of magical properties, if you ever discover it is
there. It may or may not appear in your version of Word. And if it
doesn't, you'll never know for sure that it isn't really there, on
account of it's invisible. Which is exactly what is wrong with
invisible objects and why consistency is so important. Other objects
are, strictly speaking, visible, but do not appear to be constrols, so
users, left to their own devices, might never discover their
manipulability. The secret, if you absolutely insist on one, should be
crisp and clean, for example, "you can click and drag the edges of
current Macintosh windows to size them," not, "You can click and drag
various things sometimes, but not other things other times."
Avoid uniformity. Make objects consistent with their behavior. Make objects that act differently look different.
The only way to ascertain user expectations is to do user testing. No amount of study and debate will substitute.
People cost a lot more money than machines, and while it might appear that increasing machine productivity must result in increasing human productivity, the opposite is often true. In judging the efficiency of a system, look beyond just the efficiency of the machine. For example, which of the following takes less time? Heating water in a microwave for one minute and ten seconds or heating it for one minute and eleven seconds? From the standpoint of the microwave, one minute and ten seconds is the obviously correct answer. From the standpoint of the user of the microwave, one minute and eleven seconds is faster. Why? Because in the first case, the user must press the one key twice, then visually locate the zero key, move the finger into place over it, and press it once. In the second case, the user just presses the same key—the one key—three times. It typically takes more than one second to acquire the zero key. Hence, the water is heated faster when it is "cooked" longer. Other factors beyond speed make the 111 solution more efficient. Seeking out a different key not only takes time, it requires a fairly high level of cognitive processing. While the processing is underway, the main task the user was involved with—cooking their meal—must be set aside. The longer it is set aside, the longer it will take to reacquire it. Additionally, the user who adopts the expedient of using repeating digits for microwave cooking faces fewer decisions. They soon abandon figuring out, for example, whether bacon should be cooked for two minutes and ten seconds or two minutes and twenty-three seconds. They do a fast estimate and, given the variability of water content and bacon thickness, end up with as likely a successful result with a lot less dickering up front, again increasing human efficiency.
Since, typically, the highest expense in a business is labor cost. Any time the user must wait for the system to respond before they can proceed, money is being lost.
Large organizations tend to be compartmentalized, with each group looking out for its own interests, sometimes to the detriment of the organization as a whole. Information resource departments often fall into the trap of creating or adopting systems that result in increased efficiency and lowered costs for the information resources department, but only at the cost of lowered productivity for the company as a whole. For example, one large California corporation used floppy disks as the medium for collecting benefit enrollment information. At the beginning of open enrollment, each employee would receive a disk with the enrollment applications on which he or she would insert into their computer and run. After asking for the employee’s name, address, phone number, department name, etc., the employee would be permitted to step through all the various benefits, ultimately returning the disk which now contained all their answers and decisions. The IR department then sucked the data off each disk and entered it into their system, all automatically. The IR department saved a great deal of money over the old system, where they had to key in the employee’s decisions from a paper form. What was the problem? Instead of the IR department bearing the burden of keying in the employees’ decisions, each and every employee now bore the burden of typing in his or her name, address, phone number, department name, etc. The system was just as inefficient as before, but now the cost was borne by all departments, rather than having it concentrated in the IR department’s budget.
This simple truth is why it is so important for everyone involved in a software project to appreciate the importance of making user productivity goal one and to understand the vital difference between building an efficient system and empowering an efficient user. This truth is also key to the need for close and constant cooperation, communication, and conspiracy between engineers and human interface designers if this goal is to be achieved.
Example from a fictitious word processor: Wrong:
Right: Insert:
Here, the first example, with its leading words, is actually more informative and more accurate: one does not "insert" a footnote if it is to be placed after all the other footnotes. And one does not insert a table of contents if there is already a table of contents there. Instead, one updates it. Still, the second example will prove much more efficient in time-trials. Why? Because the extra information the first example offers does not outweigh the advantage of being able to scan only the first word in each menu item to find the specific menu item you are after.
Mimic the safety, smoothness, and consistency of the natural landscape. Don’t trap users into a single path through a service, but do offer them a line of least resistance. This lets the new user and the user who just wants to get the job done in the quickest way possible and "no-brainer" way through, while still enabling those who want to explore and play what-if a means to wander farther afield.
The closer you get to the naive end of the experience curve, the more you have to rein in your users. A single-use application for accomplishing an unknown task requires a far more directive interface than a habitual-use interface for experts.
Stable visual elements not only enable people to navigate fast, they act as dependable landmarks, giving people a sense of "home."
People explore in ways beyond navigation. Sometimes they want to find out what would happen if they carried out some potentially dangerous action. Sometimes they don’t want to find out, but they do anyway by accident. By making actions reversible, users can both explore and can "get sloppy" with their work.
The unavoidable result of not supporting undo is that you must then support a bunch of dialogs that say the equivalent of, "Are you really, really sure?" Needless to say, this slows people down. In the absence of such dialogs, people slow down even further. A study a few years back showed that people in a hazardous environment make no more mistakes than people in a supportive and more visually obvious environment, but they worked a lot slower and a lot more carefully to avoid making errors.
Users should never feel trapped. They should have a clear path out.
Early software tended to make it difficult to leave. With the advent of the web, we've seen the advent of software that makes it difficult to stay. Web browsers still festoon their windows with objects and options that have nothing to do with our applications and services running within. Our task can become akin to designing a word process which, oh, by the way, will be using Photoshop's menu bar. Having 49 options on the screen that lead directly to destruction of the user's work, along with one or two that just might help is not an explorable interface, it is the interface from hell. If you are working with complex transactions using a standard web browser, turn of the menu bar and all of the other irrelevant options, then supply our own landmarks and options.
While at first glance, this law might seem patently obvious, it is one of the most ignored principles in design. Fitts's law dictates the Macintosh pull-down menu acquisition should be approximately five times faster than Windows menu acquisition, and this is proven out. Fitt's law dictates that the windows task bar will constantly and unnecessarily get in people's way, and this is proven out. Fitt's law indicates that the most quickly accessed targets on any computer display are the four corners of the screen, because of their pinning action, and yet they seem to be avoided at all costs by designers. Use large objects for important functions (Big buttons are faster). Use the pinning actions of the sides, bottom, top, and corners of your display: A single-row toolbar with tool icons that "bleed" into the edges of the display will be many times faster than a double row of icons with a carefully-applied one-pixel non-clickable edge along the side of the display. Human-interface objects are not necessarily the same as objects found in object-oriented systems. Our objects include folders, documents, and the trashcan. They appear within the user's environment and may or may not map directly to an object-oriented object. In fact, many early gui's were built entirely in non-object-oriented environments.
Latency can often be hidden from users through multi-tasking techniques, letting them continue with their work while transmission and computation take place in the background.
Eliminate any element of the application that is not helping. Be ruthless. Ideally, products would have no learning curve: users would walk up to them for the very first time and achieve instant mastery. In practice, all applications and services, no matter how simple, will display a learning curve.
Usability and learnability are not mutually exclusive. First, decide which is the most important; then attack both with vigor. Ease of learning automatically coming at the expense of ease of use is a myth.
Good metaphors are stories, creating visible pictures in the mind.
Metaphors usually evoke the familiar, but often add a new twist. For example, Windows 95 has an object called a briefcase. Like a real-world briefcase, its purpose is to help make electronic documents more portable. It does so, however, not by acting as a transport mechanism, but as a synchronizer: Documents in the desktop briefcase and the briefcase held on portable media are updated automatically when the portable media is inserted in the machine.
(Even here, it has become completely inexcusable that today's computers and operating systems do not support and encourage continuous-save. That, coupled with a small amount of power-protected memory could eliminate the embarrassment of $5000 machines offering the reliability of 10-cent toys.)
We may need to know:
and myriad other details. In addition to simply knowing where they’ve been, we can also make good use of what they’ve done.
Users should be able to log off at work, go home, and take up exactly where they left off. A private service for doctors, Physicians On Line, does an excellent job with this. Doctors can be 95% of the way through a complex transaction, log off, log in again six weeks later from another part of the world, and the service will ask them if they want to be taken right back to where they were.
Most users cannot and will not build elaborate mental maps and will become lost or tired if expected to do so. The World Wide Web, for all its pretty screens and fancy buttons, is, in effect, an invisible navigation space. True, you can always see the specific page you are on, but you cannot see anything of the vast space between pages. Once users reach our applications, we must take care to reduce navigation to a minimum and make that navigation that is left clear and natural. Present the illusion that users are always in the same place, with the work brought to them. This not only eliminates the need for maps and other navigational aids, it offers users a greater sense of mastery and autonomy. As with the inherent statelessness of the web (see Track State, above), our job is not to accept blindly what the architects have given us, but to add the layers of capability and protection that users want and need. That the web's navigation is inherently invisible is a challenge, not an inevitability.
web sites
books
|
