add a comment
Our friends over at Cyber Duck (http://www.cyber-duck.co.uk/) have been Tobii users for quite a few years now and often speak at conferences and at their own events about the way they integrate eye tracking into their user testing. Below Matthew shares a few of their thoughts on how the get the most from eye tracking and user testing…
10 ways to improve your web eye tracking studies
Web eye tracking technology offers valuable insights into how users behave when navigating a website or application. At Cyber-Duck we have been conducting our own in-house eye tracking studies since 2008; enhancing
our usability testing processes and ultimately bettering our offering as user experience (UX) specialists. At Cyber-Duck we use a Tobii T60 eye tracker hooked up to a Dell Precision M4500 laptop with Tobii Studio installed. Whilst your eye tracking tools and approach will inevitably differ from our own the tips in this article will be equally valid to your business as they are to our own.
1. Find appropriate recruits
The best kind of participants for any kind of usability studies are real users, current or future.
Whilst a client may wish to recruit participants from within their organisation for various reasons, it is important that they understand this is only appropriate if the project being tested is an internal tool for members of the organisation to use. However, if the project is a public facing website or application, testing staff members and stakeholders would not be representative of real end users and this will limit the effectiveness of the study.
It is also important to ensure diversity within test participants. If all recruits are sourced from the same organisation or profession, it is likely that they share similar user traits, which when tested could be detrimental to the findings of your studies. These similarities could result in some issues not being detected, or assumptions being made about a website based on what could be a minority of user behaviour. By testing a diverse pool which includes likely user groups (age, demographics and profession) you eliminate the risk of producing misleading data.
2. Test up to five participants
Eye tracking tests do not need to be conducted on a vast amount of participants. In most cases, provided the test has been well-designed, the first five participants will identify around 80% of usability issues. The first few participants will identify most issues with the website or application. After this, the major issues have been identified and any new issues that arise will become less frequent, and usually have less of a negative impact on user experience.
It is a good idea, if your budget allows, to consider implementing eye tracking tests throughout the project lifecycle. This allows for issues rectified following the first round of eye tracking to be tested and changes validated before the project goes live to the public.
3. Test-run your equipment
It is essential to do a full check of all equipment before the client test. You should set up all equipment and test the eye-tracking device, the computer you will be using, as well as any software and peripherals you will be relying on well in advance of the testing date. This will allow for maintenance to be conducted if any of the kit needs servicing.
On the day of the test it is always a good idea to arrive early. You want to be able to set up all equipment and have plenty of time to conduct a run through of the test to iron out any issues before participants arrive.
4. Ensure participants are relaxed
Most participants will have never taken part in eye tracking testing before, therefore it is important to ensure they are at ease before you start the test.
You should explain to them before you start that it is not them being tested but rather the system and so by making mistakes, they are actually helping you to find issues with the project. Also make it clear that you are there as an observer and not to aid them; it should prevent them from breaking their gaze from the screen and seeking assistance.
Keep task descriptions brief and simple, and refer to specific directions in a slightly abstracted manner to avoid inadvertent clues on how to accomplish the tasks. For example if as part of the test, the participant needs to sign in to their account and the button is labelled “Sign in” you could ask the user to “log in” to avoid giving away too much of a clue.
It is important to ensure that the testing environment is suitable. Ideally, the participant will forget their surroundings and their observer due to their concentration on the task at hand. If possible it is advisable to have a dedicated testing lab which promotes the optimum environment for testing. However if you have to test on-location here are some tips on how to set up the best field testing environment.
- Make sure the room is quiet
- Ensure there is an area close to the testing room for participants to wait.
- Place notices on the door to the testing room stating where participants should wait and that they should not disturb the testing environment.
- Have no more than two observers in the room with the participant and if possible have them seated out of the participants sight range.
- Do not speak and be as quiet as possible during the test.
5. Printed instructions
It is important that the participant has printed instructions of their task available to them. Whilst you should verbally introduce the task to the user before the test and have on-screen instructions at the start, printed instruction ensure the participant has a constant reference throughout. It also means that participants won’t have to seek advice or help from you to complete tasks.
6. Use real information
Encourage participants to use their real personal information when completing web forms in the test. This means that the way the user completes the form is more natural and therefore more useful when making design decisions. Dummy details corrupt the testing slightly as they aren’t a true measure of how long a form takes to complete (all this is actually accomplishing is testing the participants ability to copy information).
When participants use their own details it is far easier to identify problems with the website or application. Individuals differ in the way they enter certain data into web forms, such as telephone numbers. For example, if the dummy data presents a telephone number with no spaces, you may overlook an issue that denies users the ability to input telephone numbers with spaces. Real data can help identify these kind of web form issues. The test should consider how the system expects to receive information, how the user interprets this, and how easily the system can handle alternate formats of data.
Some users may be uncomfortable providing their genuine data. You should ensure you have consent forms ready and that these explain clearly how the participant’s data will be used and assuring them that their data will remain private and is only being used for test purposes. Ensure you have dummy data prepared in case any participants do refuse to use genuine data.
7. Take detailed notes
Eye-tracking is an extremely valuable way of collecting data about your users. However, the eye tracker will only provide data and information, which needs to be analysed and interpreted by the test initiator. The eye-tracker is unable to provide human insights regarding the data.
This is why it is essential to take detailed and comprehensive notes whilst the tests are being conducted. This enables the tester to record their own insights from observing the tests, such as what aspect of a task caused the participant to pause or hesitate. Notes should also record the participants’ personal details, such as skill level and affinity with using computers. As an observer you will also be noticing patterns in user behaviour which you can only record manually.
It is a good idea to conduct a short survey at the start of the testing session on each participant. This can help you to gain slightly more information about their skill levels and confidence using similar systems, as well as age and English language capabilities. These are all factors which can affect usability, and it is good to have these in mind when assessing participant’s results.
8. Verbal questions
It is a good idea to follow the test with some verbal questions. The participant will be able to provide valuable qualitative information regarding the product being tested whilst it is fresh in their memory.
Importantly, phrase your questions in a manner which avoids suggesting particular answers, or making assumptions about the answer. This has two benefits. Firstly it encourages the reader to describe their experience instead of giving a yes or no answer. This feedback is often more valuable. Secondly it stops you from suggesting answers to the participant. If a participant is unsure about what is expected of them they may answer with what they think you want to hear.
9. Present your findings clearly
Ensure that your findings are presented in a clear and accessible away. The client most probably will not have an in depth knowledge of terminology associated with user experience and eye tracking. Ways you can combat this are to:
- Include a glossary of any technical terms you include in your notes or recommendations. This ensures content clarity for the client.
- Include your interpretations of visuals such as gaze plots and heatmaps. The client may not understand the importance or significance of these unless it is explained.
10. Bring a designers perspective to testing
In the same way that you would bring creativity, attention to detail and empathy for participants into designing the product, you should apply these principles to your test. This will ensure a well-designed test and intelligent analysis of your results. This critical analysis will inevitably lead you to stronger solutions.
The New Tobii X2 Eye Tracker – The Smallest And Most Flexible Eye Tracker On The Market! February 11, 2013Posted by Natasha French in Advertising, eye tracking, Market Research, Marketing, Media, Shopper Research, Technology, Tobii, Uncategorized, Updates, Usability & UX.
add a comment
Acuity are proud to present the new Tobii X2 eye tracker – a ground breaking development in delivering the smallest and most flexible eye tracker on the market!
The Tobii X2-30 Eye Tracker (available in Compact Edition and Wide Edition) is a revolutionary small eye tracking system, powered by the latest generation in innovative eye technology from Tobii.
The Tobii X2 family comprises of eye tracking systems at 30 and 60 Hz. The X2 can easily be clipped on to a laptop, a PC monitor, or even a tablet for a compact and is our most portable system yet!
Research anywhere – Small footprint accommodates truly portable solutions and enables expansion of eye tracking from lab to real-life environments.
Supreme efficiency - Ease of set up and operation paired with very robust participant tracking allow for cost efficient studies.
Trust your data - Unparalleled tracking accuracy within a revolutionary large head movement box ensures reliable and valid research results.
Choose between the Compact Edition and the Wide Edition – depending on your specific study context!
The Compact Edition is a smaller version of the eye tracker, measuring 184 mm (7.3’’) in length. You can use it as your portable lab or for studies that require a small eye tracker to track what participants see on:
- Laptops and smaller PC monitors up to app. 22’’
- Tablets and mobile phones (dedicated mobile device accessories will be available soon)
- Small real-world interfaces
The Wide Edition is designed for studies that require larger gaze angles (up to 37°) and enables studies that involve larger stimuli, being able to track interfaces such as:
- PC monitors up to app. 27’’
- Projections and simulators
- Large real-world interfaces
Acuity are offering both rental and purchase options. As always for more information please contact the Acuity team at; email@example.com or (0)1189000795!
Why 2013 Is Going To Be Even Better For Our Clients…. February 5, 2013Posted by Natasha French in eye tracking, Market Research, Shopper Research, Technology, Tobii, Uncategorized, Updates, Usability & UX.
add a comment
It’s been a busy couple of years with exciting developments here at Acuity. Key to this was the launch of Acuity Intelligence just over a year ago and the addition of Technical Director Dr Tim Holmes. I’ve had a few clients contact me, curious to find out the difference between the two companies: Acuity ETS and Acuity Intelligence (ETS and AI for short!). Reflecting on 2012 and what it held for the group, I thought the New Year might be a good time to clarify the difference between both and explain why 2013 is going to be even better for our clients.
Most of you reading this will already be aware of Acuity ETS. You’ll know our Directors and hopefully come to us when you have an eye tracking sale / rental requirement. Alongside this, ETS also offer training and support, giving you the tools and ‘know how’ to complete your own independent research effectively and use scientific technology to answer difficult commercial questions.
Now, if any of you know our Directors well, you’ll also be very aware that they are technology geeks (this enthusiasm is a must for anyone that joins the Acuity team!). With a (slightly obsessive) thirst for new tech, it wasn’t long before other tools that could supplement gaze data were also found by the guys. Wanting to offer these to clients too – the idea of Acuity Intelligence began!
Customer tracking / counting technology, facial and emotional recognition solutions, non-intrusive biometric sensors, browser based eye tracking to name but a few, the supplementary offering expanded quickly along with the AI team.
The end of 2011 welcomed a new company Director and Vision Scientist, Dr Tim Holmes. Actively involved in scientific research as a university lecturer, Tim was the catalyst for AI offering additional research and consultancy services based on real science. Services which include education about why technologies such as eye tracking and galvanic skin response are important measures of consumer attention and emotional engagement; the design of rigorous and innovative research projects that maximise the value of client investment by using the right combination of methodology and technologies to answer client questions; and innovative analysis and presentation of results that are scientifically accurate, replicable and focus on the meaning of the data, rather than just the numbers themselves. With Matlab and E-Prime consultancy offerings, AI also supports experiment development and data analysis in a number of Universities in the UK and beyond and is actively involved in cutting edge research at a number of institutions. In addition to all this, AI now also has product integrators and software developers in house who are feverishly beavering away on some truly unique off-the-shelf solutions as well as bespoke solutions tailored to the needs of specific customers!
Working together for you in 2013
Both sister companies offer something very different but both work together seamlessly on your behalf to provide you with a more complete research solution for 2013; all equipment rental, technical support and analysis or simply the technology and knowledge that will enable you to complete your own independent research well. We hope that by extending our offering to you, we can offer a more flexible way to make use of new technologies that add real scientific value to your research.
Tobii Studio 3.2 Launched December 18, 2012Posted by Jon Ward in eye tracking, Studio, Technology, Tips And Tricks, Tobii.
add a comment
We are pleased to announce the latest version of the worlds most popular eye tracking analysis software. Studio 3.2 has a number of new features including task based segmentation, a new visualisation tool and a number of performance tweaks. Go to the Tobii website and download it now, if your support and upgrade contract isn’t up to date then get in touch with us and we will give you a quote so you can continue to enjoy the constant improvements and new functionality of Tobii Studio.
‘Screen Life: the view from the sofa’ Awarded The Grand Prix at The 2012 Media Research Group (MRG) Awards – Market Research Innovation Using Quividi!!!! November 29, 2012Posted by Natasha French in Market Research, Media, Technology, Uncategorized.
1 comment so far
Fantastic news that Thinkbox have been awarded the Grand Prix at the 2012 Media Research Group (MRG) Awards for its research into multi-screening with ‘Screen Life: the view from the sofa’. The study also won awards for the ‘Best Trade Body Research’ and the ‘Best Research Initiative’.
This project was carried out by our friends at COG Research for Thinkbox and is a real example of researchers pushing the boundaries of market research innovation by combining cutting edge, scientifically proven technology – such as Quividi – with market research.
‘Screen Life: the view from the sofa’ included an examination of over 700 hours of people being filmed while watching TV in their homes using Quividi, video cameras and recorders (supplied by us) to capture who was in the room and where their visual attention was focused.
Quividi is a fully automated people tracking solution that can measure the attention time of an audience using a simple video sensor and image processing technology. Acuity has seen it more commonly used in shopper research to understand the attention time of customers whilst they look at a piece of digital signage, retail fixture, POS or POP. However, it’s always exciting when we speak to commercial clients that embrace new technology and create new applications that help the industry to understand more about consumers – in the case of ‘Screen Life: the view from the sofa’ just how positive a development multi-screening is for TV advertisers.
It’s equally as good when industry leaders acknowledge those pushing the boundaries of MR innovation – and encourage others to do the same!!!
UCD2012 Conference at Cavendish Square, London… See You There! November 8, 2012Posted by Natasha French in Advertising, eye tracking, Market Research, Marketing, Media, Technology, Uncategorized, Usability & UX.
add a comment
Last weekend it was ‘The Ski and Snowboarding Show’, this weekend it’s UCD2012 Conference at Cavendish Square, London…. I suppose you could say we get around a bit!
So why UCD2012 and why Acuity? The show aims to give people the opportunity to enjoy real world case studies, inspiring presentations and hands on workshops focusing on user centred design in the real world by offering opportunities for learning and sharing with like-minded people. With UCD2012 being a not-for-profit conference for the community (it’s only been possible due to the presenters donating their time!) Acuity felt it was important to show our support by attending the event and sharing our own experiences and knowledge on the day.
With that in mind, on Saturday 10th November, Acuity will be holding interactive workshops with eye tracking and other complementary technologies we offer such as GSR, EEG and wireless biometrics – please note that if you are attending workshops are on a ﬁrst come ﬁrst served basis! Getting there early is advisable!
In addition to this, our very own Jon Ward will be giving a talk on ‘Palm Reading for The 21st Century?’ (I’ll let the mystery with that continue until Saturday!) and with talks from our friends at Cyberduck, Amberlight, User Vision, NileHQ, Sapient Nitro and Foolproof – it’s set to be an interesting and informative weekend. With our Christmas night out planned for the evening, it could be eventful too.
UCD2012 starts on Friday 9th November at 9.00am and finishes on Saturday 10th November at 5.00pm. For information and event updates please go to http://www.ucd2012.org
Your Chance to Win a Pair of Pivothead Glasses – Come See Us at Stand G77, The Ski and Snowboard Show 2012! October 29, 2012Posted by Natasha French in Glasses, Market Research, Media, Shopper Research, Technology, Uncategorized.
Tags: snow adventure
add a comment
Only 2 days until the Ski and Snowboard Show at Earls Court and I can’t wait! Scott, Tim, Alice, Nina and I are giving Acuity Intelligence a hand on stand G77 to showcase the AMAZING Pivothead Glasses!
With early starts and late finishes, it’s set to be a busy few days but with new show features, returning favourites, new experiences and spectacular stunts – oh, and did I mention a rock horror ‘Break The Ice’ party Thursday evening? It’s set to whet your appetite for your next adventure in the snow…
And what better way to capture all that snow adventure than by wearing the new Pivothead full HD 1080p hands free recording glasses available from www.acuity-intelligence.com These little gems are perfect for capturing all that daredevil snowboard and ski action and at 60 fps, you won’t miss a thing! Don’t believe me? Check it out at: http://goo.gl/1JHrZ Even better, come along and see us at Earls Court from 31st October to 4th November for your chance to win a set of your own Pivotheads!
False Memories…. October 1, 2012Posted by Natasha French in Advertising, eye tracking, Market Research, Shopper Research, Technology.
1 comment so far
Tim Holmes, Technical Director at our sister company Acuity Intelligence, is an avid blogger and he’s inspired me. He’s put my lack of blogging to shame so without further ado, I’m going to write my first (of many) blogs – beginning with an entry prompted by forensic psychologist Scott Fraser and a TED talk he recently gave that considers ‘Why Eye Witnesses Get It Wrong’.
Although centred on a criminal case, this talk illustrates from a different perspective why using memory recall alone can show itself to be an unreliable way of gaining evidence. It also demonstrates why it’s so important that behavioural measures and analysis techniques are used to help determine the veracity of a particular memory.
Fraser attributes false memory as ‘the reason why eye witnesses get it wrong’. He explains that we can’t cope with all the sensory input so we filter it based on what we think is important at the time. This is what attention is! So when something becomes important after the fact, it isn’t necessarily in our memory at all.
A great example of this from eye-witness world is something called ‘weapon focus’ which means that if there’s a gun present, the witness tends to focus all their attention on the gun (the thing that they THINK is a threat to them) rather than the person holding the gun (the thing that is ACTUALLY a threat to them)so in the case of Fraser’s case study, the witness only has a partial story, and with no requirement for any motivation processing, the brain then is then filled with information that wasn’t actually stored in the first instance. On reflection, I can think of many instances where I’m sure something has happened, only to be told by my husband, ‘’that’s not the case’’ (I’m SURE I told him I was going to buy that new pair of shoes….!)
So, back to Fraser’s point about false memory, if a subject is recalling an experience that they believe to be a truthful statement of events, it could still be inaccurate and misleading. In a commercial context, there’s nothing to suggest that this situation would be different if a subject were asked to recall a shopping experience and recall ‘why’ their attention was drawn to a particular pack or after being asked to ‘think about the last time they bought an item’. This was illustrated by a classic article in Psychological Review ‘Telling More Than We Know; Verbal Reports on Mental Processes’ by Richard E.Nisbett and Timothy DeCamp Wilson at the University of Michigan. In this paper, cognitive psychologists Mandler, Miller and Neisser propose that ‘we have no direct access to order mental processes such as those involved in evaluation, judgement, problem solving and the initiation of behaviour’.
Frasers talk doesn’t surprise me. It’s been suggested that 95% of all decisions we make are subconscious with Gerald Zaltman of Harvard Business School and other psychologists supporting this idea and it makes sense that if most of our behaviour is subconscious and the brain tries to fill in any gaps with what it thinks is most likely based on experience. Eye tracking and physiological measures give you specific metrics which are derived from that subconscious decision making processes in addition to a consciously expressed opinion or pieced together memory. In other words, together with the consumers self-report, they give you a much more complete picture.
On a closing note, irony would have it that during Fraser’s talk there was inaccuracy in one of Fraser’s comments about the Twin Towers, clearly illustrating a faulty example testifying a good theory.
A talk on false memory with a false memory is somewhat ironic but does prove a point!
add a comment
Acuity are proud to announce their latest partnership with French company, TEA. By adding their products to our portfolio we can now offer a range of sensors to monitor GSR, sEMG, heart rate, acceleration, orientation and gornio / torsio movements – and more! Utilising a simple but powerful interface the Captiv range of software allows multiple data sources (video, EEG, eye tracking, physiological sensors, etc) to be synchronised and analysed using powerful a coding mechanism giving you instant access to the data.
The Captiv L2100 software allows post event coding of a video stream and is ideal for sports science, ergonomics, health & safety, zoological studies and more – allowing the user to quickly isolate events, transitions and elements of interest. It can also be used to analyse data captured with the L7000 software which opens up great opportunities for commercial and academic institutes to capture and share data and equipment accordingly.
The L7000 software allows observation and analysis across a wide range of platforms and data sources – including live integration with Tobii and FaceLab eye tracking units. Post synchronisation of almost any other eye tracking platform is also possible meaning this is a great addition to any research facility. As well as offering the powerful coding tools of the L2100 software, you can also add the T-Sense range of sensors to your projects, meaning GSR, heart rate, FSR, sEMG and more can be captured and synchronised to your other data streams – including up to 10 video sources! This gives you the power and flexibility to observe a user from various angles, monitor multiple users in an environment, study team dynamics or measure a user physiological state during computer gaming… and more! With the capabilities to record locally to a PC or to the T-Log recording device (which is around the size of a 1st generation iPod) the system is quick to deploy and incredibly portable.
As you can see from the image – the sensors are incredibly small and lightweight, so the user is barely aware of them during testing. The system is fully customisable and additional sensors can be bought ad hoc to suit specific requirements. We have recently used these sensors for projects with the National Geographic channel, BBC Radio 4 and Oxford University and have been massively impressed with the flexibility, data and capabilities of the system.
Please contact us for more details or to discuss specific requirements, or to arrange a demonstration – firstname.lastname@example.org
Tobii Studio 3.0.2 Available to Download February 20, 2012Posted by Jon Ward in eye tracking, Studio, Tobii, Updates.
add a comment
For all our customers with current support and upgrade contracts please go to http://studiohelp.tobii.org/Updates/ with your licence key handy – this minor release tweaks some of the functionality released in Studio 3.0. For those without a support contract please get in touch with us at email@example.com to discuss your options.