WHAT IS IT?
AND HOW do i use it in report writing?
I’m often asked what method I use to categorise and report the severity of usability issues, and as such have included a section in my UX Research 101 Training Programme that covers all things usability, including this topic.
Over the years I have developed my own usability issue matrix, a system that I use to ensure rigour in my analysis (mitigating against my own biases), whilst providing clarity in my report writing and communication of usability testing results.
“A usability problem is an aspect of the system and/ or a demand on the user which makes it unpleasant, inefficient,
onerous or impossible for the user to achieve their goals in typical usage situations.”
The system enables me to rank usability issues in order of importance based on the frequency and severity of the issue that has been observed.
I classify usability issues by asking the following three questions:
Does the issue occur on a core user journey?
Is the issue difficult for the user to overcome?
Is the issue persistent?
Let’s go into these questions in a little more detail.
A core user journey is one that is essential to your product’s function and supports your business’ revenue stream, for example your check out or purchase process.
Issues that are difficult for users to overcome usually occur when there is no other way for the user to complete the task they’re doing, or for example if they get stuck in a loop or discover a dead end.
An issue is persistent if it continually affects the user trying to achieve their goals.
These are binary choice questions, you are required to answer either yes or no, and the number of ‘yes’ answers you get for each issue will determine the severity of the issue you are dealing with.
A great way to visualise this is using a decision tree:
If you answer yes to all three questions, you have a severe usability issue. You need to draw this issue to your team’s attention quickly ensuring the problem is addressed immediately.
Conversely, if you answer no to all three questions, the issue you have observed will be classified as a low priority and so does not demand as much attention. However, it is worth to note that although medium and low priority issues might not need immediate attention, these are still usability issues, and do need to be considered (and definitely not ignored!). Too many low issues will negatively affect credibility and may damage brand reputation, as well as reduce the user’s overall satisfaction with the product, impacting the overall user experience.
Communicating the severity of the issues you have identified to your team is important, and so I tend to find clear labelling is key to drawing attention where needed.
I use the following icons and labels to give a more visual description to my reports:
I find its also useful to be specific about where, and with what element the usability issue stems from.
Usability problems may include (not an official or exhaustive list):
- Visual elements (e.g. a button not appearing to be clickable, or having an ambiguous symbol depicting its underlying function)
- Navigation (e.g. inconsistent back button placement from page to page)
- Language (e.g. unfamiliar or indecipherable terms for actions)
- Layout (e.g. an important page element may be placed in such a way that the users totally miss it)
- Accessibility (e.g. inadequate contrast ratios, textual alternatives to images, lack of support for sole keyboard/ screen reader use)
Using a combination of specific and accurate descriptions of both the severity, and the element of the design causing the issue, our research reports become clearer. Feedback from Product Owners and UX Designers on this method have been positive, and having clear coding in the form of icons and descriptions has proved useful in prioritisation exercises after testing is complete.
I hope you have found this introduction to The Usability Issue Matrix useful. If you would like to learn more about usability testing and user research methods and techniques, please take a look at the details for my UX Research 101 Training Programme.
Usability testing in the Zoopla in-house UX Lab