How to minimize availability bias during user studies or interviews?

Decorative image showing the text Dear Diary — Graphic by author

What is availability bias and how to deal with it?

As Kahneman and Tversky showed, biases are human. And we may be ready to listen as user researchers but human memories are fallible. In a user study, people’s answers may suffer from ‘availability bias’. That is, people answer based on what first comes to mind rather than what’s most important to them. How do we deal with this?

Biases are generally shortcuts to help us think faster, so slowing down the thinking process is a good way to address several biases. For dealing with availability bias, below are the main approaches that I have learned over the years:

If possible, give the participants more time to reflect on the topic before or during the study. This can help slow down the thinking process, giving participants a chance to look through their mental, digital, or physical records. And to go beyond the first thing that comes to mind.
Secondly, enable the participants to express themselves in more than one medium. For example, by speaking, writing, showing photos, drawing, showing/recording video, audio, etc. Using more than one medium activates different centers in our brain, which may bring up more memories or more details. It also enables participants who prefer different ways of expressing themselves, instead of talking.
Finally, use methods that do not rely (only) on the memory of the participants, for example, collecting data on or observing the real-time behavior of people.

Example approaches for minimizing availbility bias

Here are a few examples of approaches to give participants more time, using multiple methods, and using more than one medium of expression.

Mix methods

Mixing different methods, for example, combining a diary study with interviews can help avoid biases and collect rich data. Last year, we conducted a study where we compared data from only interviews vs. interviews + diary study. We found that a diary study helps people reflect on a topic for a longer time. It also enables you to use various mediums – photos/videos/text to explore users’ experiences. And this gets you more relevant data with specific and richer examples.

Allow participants to express themselves in multiple mediums

You can ask participants to think about the topic of your study in advance and ask them to bring some photos/audio/video data representing their thoughts. Then you can discuss those during the interview.Just like I outlined my process here, I would suggest you to do that same, to get an insight into your process of working, so you can apply it everywhere :).

Activity-based approach

Meena Kothandaraman from twig + fish shared this research approach. They use an activity as a ‘hook’ into the conversation with people. For example, they start the interview by asking the users to fill an empathy map around the topic. This way people get some time to gather their thoughts. Also, it gives a voice to people who prefer to write or draw more instead of talking. They can lead you through their map and thus feel more in control of the conversation. Below is an image of an empathy map with questions such as: What do they think and feel, or see, or say and do, and hear? Where are the pains and the gains?

P.S. Thank you for reading. To receive these posts in your inbox weekly, you can join hundreds of researchers here: https://tinyurl.com/altuxrsign