Expert usability reviews – How to get the best value

19 December 2011
Ben Logan

Ben Logan


Passionate about improving services and experiences for the people that use them.

Reviews require a firm grasp of the design problem and should precede usability testing. They should also: use multiple experts, include other relevant designs (including competitors’) and strongly communicate their results.

What is an expert usability review?

An expert usability review is one of the best – and lowest cost – methods for identifying usability issues with a design. An expert usability review involves a usability expert using their skills and experience to personally evaluate how well a design will support its target audience(s) in achieving their specified goal(s) with effectiveness, efficiency, and satisfaction (within a specified context of use).

The usability expert will conduct their review according to a set of generally accepted usability guidelines (also known as heuristics), as well as their personal knowledge of the design’s domain (the area of knowledge/expertise for which the design is intended).

back to top

Defining an expert usability review’s design problem

As discussed above, in order to offer the best value an expert usability review must focus on the efficacy with which a design achieves its aim. As such, we must clearly define a design’s intended audience(s), goal(s) and context(s) of use.

If the usability expert conducting a review does not have a firm and accurate understanding of these factors, they will be forced to apply usability guidelines in a rather general way. Such an application of usability guidelines is likely to miss many usability issues specific to the design’s intended audience(s), goal(s) and context(s) of use.

Because of this, we strongly recommend that every effort is made to communicate an accurate and firm understanding of these factors to the usability expert. We have found that clear communication can often be supported by detailed personas and scenarios (which may have been previously used to support the design process).

We have found that a usability expert can usually learn enough about the domain (the area of knowledge/expertise for which the design is intended) through learning of its intended audience(s), goal(s) and context(s) of use. It is, however, the case that designs for particularly complex domains may require additional explanation and background information.

back to top

Which usability guidelines will be used?

It can sometimes be helpful to provide an overview of the kind of usability guidelines which will be used within an expert usability review. Providing a complete list of all the possible usability guidelines that could be used is likely to be impractical, but an overview can help set expectations and focus the expert reviewer’s attention.

Such an overview could, for example, identify the areas of the site’s design (for example: Homepage design, Navigation, Copywriting and Forms) and/or the usability principles (for example: User expectations, Internal consistency, and User feedback) to be covered.

back to top

When to use an expert usability review

We have found that if expert usability evaluations and usability testing sessions (which involves the real-life audience using the design under observation) methods are conducted on the same design, they will identify many of the issues – but they will almost always identify some different usability issues as well. As such, almost every design project should incorporate both expert usability evaluations and usability testing sessions.

In general, the expert usability review tends to identify the less domain-specific usability issues (which one would expect from a largely guideline-based review), whereas the usability testing sessions tend to identify more domain-specific usability issues. We would usually recommend that the expert usability review should be conducted first. This will allow the more ‘basic’ usability issues to be corrected, so that the (more expensive) usability testing sessions can focus on the more domain-specific usability issues.

back to top

How many usability experts do we need?

In our experience, no single individual usability expert will ever produce a perfect review which captures all the possible usability issues with a design. Indeed, it is very rare that a single usability expert’s review will even capture all the major usability issues with a design.

This is, of course, because human beings are imperfect and no single usability expert can ever be relied upon to always indentify all of a design’s usability issues within their review. As such, we would always recommend that an expert usability review should use multiple evaluators in order to guarantee that as many usability issues as possible are captured.

In our experience, the best results come from between 3 – 5 usability experts. We have found that this will normally capture the majority of the usability issues with a design. If one employs more experts than this, the law of diminishing returns soon applies.

back to top

Should usability experts work together or alone?

We strongly recommend that all the usability experts should work alone on their expert usability reviews. Of course, their individual findings will need to be brought together into a final document – but their actual evaluations and initial write-ups should be done in isolation.

This is because we have found that running such multiple expert reviews separately provides better results than having the experts work together. Specifically, we have found that more usability issues will be identified with a design if the expert usability review is run in this way.

back to top

Other designs to consider in the expert usability review

As well as the specific design under consideration, it may be worthwhile considering whether or not any other designs should be considered. This can be valuable in at least two ways:

User expectations and consistency
If the audience can be expected to be familiar with other designs which might set some expectations for the design currently under review (the ‘evaluation-design’), then it would probably make sense to include these other designs in the expert usability review. The aim of this would be to make sure that the ‘evaluation-design’ did not conflict and/or was not inconsistent with the other designs.

Taking inspiration
Competitors’ designs can be a great source of ideas on how to solve certain design problems. Leveraging others’ designs in order to improve the evaluation-design can be a very valuable exercise.

The number and type of designs included within the overall expert usability review will, of course, be dependent on the nature of the project. In our experience, however, including such factors within the expert usability review can deliver significant benefits.

back to top

Contents of an expert usability review

An expert usability review should have several important characteristics in order for it to deliver the most helpful information to the design team. The first of these is that it should clearly state and explain each of the individual usability issues. The usability expert conducting the review can not assume that the review’s audience will be familiar with any of the usability guidelines used to conduct the review. As such, any criticism must be explained in significant detail.

Another important consideration is that the usability expert review should assign a level of priority to each of the usability issues identified. Without any prioritisation, all of the issues within an expert usability review appear to be equal – which is almost never the case. The usability expert should provide a clear indication of each issue’s priority that is based on its importance to the design’s intended audience(s), goal(s) and context(s) of use.

Most expert usability reviews will also include one or more design recommendations for each usability issue. These recommendations might be very specific and include and element of explicit re-design (such as wireframing), or they may simply suggest design options and/or approached which might be suitable.

back to top

Communicating results of an expert usability review

We would normally recommend that significant effort is made to communicate an expert usability review’s findings. Although a full document may be necessary in order to fully capture these results, it is rarely the best medium for their communication and proliferation throughout an organisation.

We have found that presentations are very good at helping design teams (and other stakeholders) understand a review’s findings. It is worth noting that getting buy-in from the relevant stakeholders is essential if the expert usability review’s findings are going to be acted upon. As such, the effective communication of an expert usability review’s results is essential.
Summary – getting value from expert usability reviews

To get the best value from an expert usability review, it is vital that the target audience(s), goal(s) and context(s) of use are clearly defined and communicated to the usability expert conducting the review. For particularly complicated domains (the area of knowledge/expertise for which the design is intended), some extra information may be required.

Clearly stating the guidelines to be used within the expert usability review will help set stakeholder expectations. Most design projects should involve both expert usability reviews and usability testing sessions. In general, the expert usability reviews should be used to eliminate any basic usability issues with the design before the more domain-targeted usability testing sessions.

We would recommend that between the 3-5 usability experts conduct individual expert usability reviews, in order to make sure that most major usability issues are identified.

Finally, value can only be delivered from an expert usability review if the relevant stakeholders understand, accept and address its findings. As such, presenting an expert usability review’s findings in an effective way is critical.

back to top

Ben Logan - Director

Got a project in mind?

Ben is on hand to answer your questions.