add a comment
Our friends over at Cyber Duck (http://www.cyber-duck.co.uk/) have been Tobii users for quite a few years now and often speak at conferences and at their own events about the way they integrate eye tracking into their user testing. Below Matthew shares a few of their thoughts on how the get the most from eye tracking and user testing…
10 ways to improve your web eye tracking studies
Web eye tracking technology offers valuable insights into how users behave when navigating a website or application. At Cyber-Duck we have been conducting our own in-house eye tracking studies since 2008; enhancing
our usability testing processes and ultimately bettering our offering as user experience (UX) specialists. At Cyber-Duck we use a Tobii T60 eye tracker hooked up to a Dell Precision M4500 laptop with Tobii Studio installed. Whilst your eye tracking tools and approach will inevitably differ from our own the tips in this article will be equally valid to your business as they are to our own.
1. Find appropriate recruits
The best kind of participants for any kind of usability studies are real users, current or future.
Whilst a client may wish to recruit participants from within their organisation for various reasons, it is important that they understand this is only appropriate if the project being tested is an internal tool for members of the organisation to use. However, if the project is a public facing website or application, testing staff members and stakeholders would not be representative of real end users and this will limit the effectiveness of the study.
It is also important to ensure diversity within test participants. If all recruits are sourced from the same organisation or profession, it is likely that they share similar user traits, which when tested could be detrimental to the findings of your studies. These similarities could result in some issues not being detected, or assumptions being made about a website based on what could be a minority of user behaviour. By testing a diverse pool which includes likely user groups (age, demographics and profession) you eliminate the risk of producing misleading data.
2. Test up to five participants
Eye tracking tests do not need to be conducted on a vast amount of participants. In most cases, provided the test has been well-designed, the first five participants will identify around 80% of usability issues. The first few participants will identify most issues with the website or application. After this, the major issues have been identified and any new issues that arise will become less frequent, and usually have less of a negative impact on user experience.
It is a good idea, if your budget allows, to consider implementing eye tracking tests throughout the project lifecycle. This allows for issues rectified following the first round of eye tracking to be tested and changes validated before the project goes live to the public.
3. Test-run your equipment
It is essential to do a full check of all equipment before the client test. You should set up all equipment and test the eye-tracking device, the computer you will be using, as well as any software and peripherals you will be relying on well in advance of the testing date. This will allow for maintenance to be conducted if any of the kit needs servicing.
On the day of the test it is always a good idea to arrive early. You want to be able to set up all equipment and have plenty of time to conduct a run through of the test to iron out any issues before participants arrive.
4. Ensure participants are relaxed
Most participants will have never taken part in eye tracking testing before, therefore it is important to ensure they are at ease before you start the test.
You should explain to them before you start that it is not them being tested but rather the system and so by making mistakes, they are actually helping you to find issues with the project. Also make it clear that you are there as an observer and not to aid them; it should prevent them from breaking their gaze from the screen and seeking assistance.
Keep task descriptions brief and simple, and refer to specific directions in a slightly abstracted manner to avoid inadvertent clues on how to accomplish the tasks. For example if as part of the test, the participant needs to sign in to their account and the button is labelled “Sign in” you could ask the user to “log in” to avoid giving away too much of a clue.
It is important to ensure that the testing environment is suitable. Ideally, the participant will forget their surroundings and their observer due to their concentration on the task at hand. If possible it is advisable to have a dedicated testing lab which promotes the optimum environment for testing. However if you have to test on-location here are some tips on how to set up the best field testing environment.
– Make sure the room is quiet
– Ensure there is an area close to the testing room for participants to wait.
– Place notices on the door to the testing room stating where participants should wait and that they should not disturb the testing environment.
– Have no more than two observers in the room with the participant and if possible have them seated out of the participants sight range.
– Do not speak and be as quiet as possible during the test.
5. Printed instructions
It is important that the participant has printed instructions of their task available to them. Whilst you should verbally introduce the task to the user before the test and have on-screen instructions at the start, printed instruction ensure the participant has a constant reference throughout. It also means that participants won’t have to seek advice or help from you to complete tasks.
6. Use real information
Encourage participants to use their real personal information when completing web forms in the test. This means that the way the user completes the form is more natural and therefore more useful when making design decisions. Dummy details corrupt the testing slightly as they aren’t a true measure of how long a form takes to complete (all this is actually accomplishing is testing the participants ability to copy information).
When participants use their own details it is far easier to identify problems with the website or application. Individuals differ in the way they enter certain data into web forms, such as telephone numbers. For example, if the dummy data presents a telephone number with no spaces, you may overlook an issue that denies users the ability to input telephone numbers with spaces. Real data can help identify these kind of web form issues. The test should consider how the system expects to receive information, how the user interprets this, and how easily the system can handle alternate formats of data.
Some users may be uncomfortable providing their genuine data. You should ensure you have consent forms ready and that these explain clearly how the participant’s data will be used and assuring them that their data will remain private and is only being used for test purposes. Ensure you have dummy data prepared in case any participants do refuse to use genuine data.
7. Take detailed notes
Eye-tracking is an extremely valuable way of collecting data about your users. However, the eye tracker will only provide data and information, which needs to be analysed and interpreted by the test initiator. The eye-tracker is unable to provide human insights regarding the data.
This is why it is essential to take detailed and comprehensive notes whilst the tests are being conducted. This enables the tester to record their own insights from observing the tests, such as what aspect of a task caused the participant to pause or hesitate. Notes should also record the participants’ personal details, such as skill level and affinity with using computers. As an observer you will also be noticing patterns in user behaviour which you can only record manually.
It is a good idea to conduct a short survey at the start of the testing session on each participant. This can help you to gain slightly more information about their skill levels and confidence using similar systems, as well as age and English language capabilities. These are all factors which can affect usability, and it is good to have these in mind when assessing participant’s results.
8. Verbal questions
It is a good idea to follow the test with some verbal questions. The participant will be able to provide valuable qualitative information regarding the product being tested whilst it is fresh in their memory.
Importantly, phrase your questions in a manner which avoids suggesting particular answers, or making assumptions about the answer. This has two benefits. Firstly it encourages the reader to describe their experience instead of giving a yes or no answer. This feedback is often more valuable. Secondly it stops you from suggesting answers to the participant. If a participant is unsure about what is expected of them they may answer with what they think you want to hear.
9. Present your findings clearly
Ensure that your findings are presented in a clear and accessible away. The client most probably will not have an in depth knowledge of terminology associated with user experience and eye tracking. Ways you can combat this are to:
– Include a glossary of any technical terms you include in your notes or recommendations. This ensures content clarity for the client.
– Include your interpretations of visuals such as gaze plots and heatmaps. The client may not understand the importance or significance of these unless it is explained.
10. Bring a designers perspective to testing
In the same way that you would bring creativity, attention to detail and empathy for participants into designing the product, you should apply these principles to your test. This will ensure a well-designed test and intelligent analysis of your results. This critical analysis will inevitably lead you to stronger solutions.