Think Aloud Protocol User Testing – Findings and Solutions

Think Aloud Protocols are designed to highlight the strengths and weaknesses in a system by encouraging participants to voice their thoughts as they navigate through the system. After conducting a think aloud experiment with four participants, there were elements of our system that caused confusion amongst multiple participants. These elements of our UI may not be designed in the best possible way to ensure a positive user experience. The aim of these experiments was to discover what elements of our system need to be redesigned so that we can offer the best user experience possible and ensure navigation through our application is easy for the user.

There were a few elements of our interface that caused participants difficulty, one of which was the select character screen. One user attempted to press the characters name as opposed to her avatar in order to select the character. This caused confusion as the interface did not react to this interaction. This is a simple design flaw and can be easily fixed by adding a link to the characters name, that way either the avatar or character name will allow the user to progress to the next screen.

A prominent issue that occurred during the experiments was users overlooking the read button that displays the informational text, a few participants had navigated through a few screens before they realised that the button was important to gain the information and put the imagery into context. As a default we had designed the application so that the text div was collapsed, and muted. To fix this design issue the read button will by default have the text on-screen and the button can then collapse it. The voiceover will also play automatically. These design changes will ensure that users are aware of the information text from the beginning as it is of huge importance within the app.

For one participant it took a few screens until they realised there was an interaction when they tapped the image on every page. The user did not press the image and see that they could interact with it, they were a few pages in before they realised that was an option. To fix this issue on the first page with an interaction, the informational text will instruct users to interact with the imagery on screen. Any interactions or animations that can be played on loop to add interest to the imagery should also play without a trigger. This will allow for the screen to seem more interactive and less static.

The final issue that occured with participants was confusion between the ovulation and menstruation screens, as the imagery is similar aside from the position of the egg on screen, one user initially navigated past the second screen assuming it was the same as the first. To solve this issue the background colour on the second screen will be changes so that it is different from that on the first. This will ensure that it is obvious to users that the screens are different. Going forward we will also ensure that all consecutive screens have different background colors.

Think Aloud Protocol – User Testing Study Design


To test the UX of our application a think aloud protocol was completed, concerning the prototyped section of the application. Participants were asked to complete the tasks below using the application provided to them on our tablet, and think aloud as they are doing so.

Why Think Aloud ?

“Think Aloud (TA) studies provide rich verbal data about reasoning during a problem solving task. Using TA and Protocol analysis, investigators can identify the information that is concentrated on during problem solving and how that information is used to facilitate problem resolution.” (E. Fonteyn, Marsha & Kuipers, Benjamin & Grobe, Susan. 1993). This method involves providing participants with a task and asking them to think aloud as they attempt to complete the task. Participants verbalize their thought patterns as they navigate through the system, and from this we gain an insight into what a user who is unfamiliar with the system experiences when the interact with our UI. The reason this is important is because as designers we have become familiar with the system we are developing and we may often overlook details that may cause confusion to our users. “With the rise of cognitive psychology over the past 30 years, thinking aloud has again become a standard method for studying human thought.  The conditions under which it provides access to silent thoughts are now believed to be well understood..” (Dumas, Joseph 2001).

The purpose of this think aloud study for us was to gain an understanding into how the developed UI is interacted with by users. The aim was to gauge what elements of the application may cause individuals confusion, were users able to navigate through the system, and what elements of the system did people think were effective. We completed this study on four participants, notes were taken while each person completed the task and a screen recording was also captured to record the users movements through the system.

The Study Design

Research Methodology : This study takes a qualitative approach and aims to gather rich data about ‘Let’s Talk’ prototype through the use of think aloud protocol. Participants will be asked to complete a series of tasks with the Let’s Talk Prototype and think aloud as they attempt to complete these tasks.

Methods : Think Aloud Protocol, this allows researcher to gather rich verbal data on a users thought process during the completion of a set of tasks.

Location: The location of this study will be in an environment that is private and comfortable for the individual participants, be it their own home or a classroom, where they can speak freely without fear of being overheard. The task will be completed on a Alba Nou Tablet, provided by the researcher.  

Participants:

Population:  We will be focusing this study on people above the age of 18 due to the sensitive content in the application

Sample Size: As Think Aloud Protocol will provide us with rich qualitative data we conducted this study with a sample size of 4 participants.The participants may have a range of technical abilities, ages and genders.

Study Procedure : Participants will be asked to complete a set of tasks using the ‘Let’s Talk’ application prototype and think aloud while they are doing so.

Please complete the tasks below and think aloud as you are doing so..

  1. Open Let’s Talk Application
  2. Login Using these details….

Username: username

Password: passcode

  1. Select Rosin as your Character
  2. Navigate through the application until you reach the section on period pain

Data gathering : Notepad and paper to take notes of what participant is saying and a screencast to record their movements on the application interface

What was gathered ?

Screencasts :https://photos.app.goo.gl/ULiYdgd5D9gACPcQ6ts

Notes: (See Blogpost on Data Analysis)

References

Dumas, Joseph. (2001). Usability testing methods: Think-aloud protocols. 22/04/2019 via URL https://www.researchgate.net/publication/319130250_Usability_testing_methods_Think-aloud_protocols

E. Fonteyn, Marsha & Kuipers, Benjamin & Grobe, Susan. (1993). A Description of Think Aloud Method and Protocol Analysis. Qualitative Health Research – QUAL HEALTH RES. 3. 430-441. 10.1177/104973239300300403, accessed  22/04/2019 via URL https://www.researchgate.net/publication/249674934_A_Description_of_Think_Aloud_Method_and_Protocol_Analysis