Over the last 10 years, practitioners have been debating whether and how eye tracking fits with usability research. But how can you compare a method with a technology? Perhaps if we understand some of the real issues of this debate, you’ll be in a better position to find the right method, tools and media to measure your user experience.
You can get the same results at a fraction of the cost.
A colleague once told me traditional usability testing can uncover as much as 95‰ issues at a third the cost of eye tracking. While this certainly can be the case, a competent practitioner using an expert review can pick up 75‰ of the issues at a quarter of the cost of usability testing from an expert review. (e.g.http://www.alistapart.com/articles/the-myth-of-usability-testing). On the other hand, it’s easy to dismiss an expert review as one person’s opinion. Watching real users uncovering the same issues is harder to challenge. Seeing where people look is more engaging to watch, limits conjecture, and translates into shorter reports.
Eye tracking can be done using many methods.
It has also been said that eye tracking has changed the way practitioners apply user testing. The reason, they tell us, is users in an interview look in a certain direction when asked a question, affecting the users gaze path and ultimately the heatmap. Getting people to first complete tasks, and then afterwards asking about their experience helps get around this. However, using larger sample sizes, which is generally recommended with eye tracking, counteracts this affect, but, you can use as long as you explain the results.
Eye tracking isn’t just about heatmaps.
Recently we completed a series of fortnightly user tests to fit within an agile development framework, for a sign-on process. We tested 6-8 people per round. Eye tracking effectively showed how certain layout options of colour and iconography made instructions and error messaging clearer. The gaze replay quickly made the point without need for a report.
What is eye tracking good for?
Usability testing focuses on users completing tasks. Fine, but what about brand awareness, key message positioning, and the users awareness of promotions or advertising? After all, these are what generate revenue, and this is where eye tracking comes into its own.
Eye tracking is best put to use when evaluating visual effectiveness such as:
- Comparing page options – looking at the how page layout and colour scheme highlights content.
- Assessing page flow – finding out how people look for information; prospective visitors normally scan an entire page, whereas current customers go straight to favourite sections.
- Attention and perception – showing whether key content like brand values, strap lines, promotions, or advertising has been or could have been noticed.
- Fast turnarounds – the richer the presentation, the less need for reporting; gaze replays remove the need for interpretation and long explanations.
- Benchmarking – to show your staff how customers use your product, or to benchmark your product, gaze paths underscore what might be a dull user experience.
While eye tracking indicates what’s been looked at, it doesn’t show if it’s been noticed, because most information is processed unconsciously and indirectly (called peripheral perception). This can be measured directly using a survey before and after a study or indirectly such as the Implicit Association Test. More recently, people’s reactions can be measured on the spot by analysing the the brain’s Electroencephalography (EEG) wave patterns.
Head candy is the new eye candy
We are moving into a new era in customer research which goes beyond merely completing tasks. Welcome to the world of neuromarketing. Now, the user isn’t asked questions or required to complete tasks. They are simply monitored, and then EEG frequencies measure responses indicating attention, concentration, and emotion, connected with an eye tracker to whatever they’re looking at. Surely this technology can be another effective tool in the user experience suite of options.
What lies ahead?
Eye tracking itself is entering a new era. Soon it will take on wearable and invisible forms, such as being embedded in eyeglasses, making it ideal for tracking user experience beyond the computer desk and into mobile research, and ultimately, into the greater world. Of greater interest, will be the opportunities for augmented reality where eye tracking technology becomes a key input device.
Conclusion – The facilitator counts most.
Technology has always influenced the way people conduct research, from Dictaphones to high-tech research labs. More importantly, it can affect the way the person feels and behaves. The skills of the interviewer can have a great impact on whether the participant experiences the research technology as inhibiting or irrelevant.
Using tools, such as screen capture, webcams, one way mirrors, and eye tracking, or media such as picture in picture, gaze path, and heatmaps are good techniques to engage an audience. And though an expert may not need these devices to know what the issues are, the client might. Remember they facilitators have different skills, so be sure to work out the purpose of the study, the skills and experience of the audience, and the amount of time they have to digest and discuss the findings.
Finally, if you’re investing in user research for staff training, or for customer closeness, the cost benefit of eye tracking becomes a valuable proposition among the many tools available to you. Here are some questions to help you decide which methods, tools, and media are best for you.
Below are some questions to help you decide which methods, tools and media are appropriate for your situation.
|Do I need to find out issues and key recommendations quickly?||Expert Reviews will uncover many issues quickly at a fraction of the cost.||It’s a good idea to conduct an expert review before user research so you can focus on the right issues.|
|Do I need to prove these results to a wider audience?||Test with a significant number of real users and encourage as many people to attend as possible.||Consider an expert review or informal testing with a small set of users; they usually reveal unforeseen issues.|
|Is the audience time poor?||Present back issues in PowerPoint bullets, and include video clips.||Consider presenting findings and following them with a solutions workshop, rather than creating a detailed document.|
|Do I need to show my colleagues what it’s like to be a customer?||Try formal testing in a usability lab with screen/video capture and picture-in-picture highlights.||Conduct informal testing, using a spare room, office, or cafe.|
|Are you working in an Agile framework?||Test regularly (every two weeks) with 3-6 users. Consider screen/video capture. Eye tracking is optional. Limit the report to a simple list of issues.||Carry on conducting user testing when there is something useful to test.|
|Do I need to benchmark this product against future development?||Test with a large sample of users (12-50+) with predefined tasks so you can measure performance quantitatively (summative testing).||Test with 4-8 users and a smaller set of tasks which you can change as the testing progresses.|
|Is the design and marketing messages important?||Use eye tracking to show whether items were looked at, and a survey to find out if attitudes have changed. Use gaze replays and heat maps to highlight issues.||If task completion is important, record the number of people that complete tasks including backtracks.|
|Do I need to assess brand awareness or advertising/promotion awareness?||Consider eye tracking with an EEG to see if page elements have people feel different or make them have to think more.||Standard usability testing is fine.|