How to use your user research questions and research frameworks to choose your research approach and methods?

My thoughts on Meena Kothandaraman's words, "Don't be led by the method, be led by your questions." This one is a foundational post on user research as it ties together (research) questions, research frameworks, user research methods, and examples. I used the Ncredible framework by twig+fish research practice to bring together all these. Many thanks to Meena for patiently walking me through her thoughts and their framework.
The Ncredible research framework by twig + fish. Credit: https://www.twigandfish.com/

Do you ever get briefs like, “Could you run a few interviews? Could you conduct a usability test?” Most of us are familiar with such briefs and sometimes executing this brief might work. But sometimes executing such a method-led brief can also lead to unrealistic expectations from the user study, or sub-optimal usage of all the time and resources, poor or no follow-up, and even getting invalid or irrelevant results.

Because these briefs are just methods, and there are hundreds of methods out there (see book reference at the end of the email). Methods alone say nothing about the purpose of the study, and without a purpose, it is difficult to define what success looks like. And if we don’t know what success looks like, it is difficult to succeed, except by chance. What do we do then?

Start with the questions

A quote by Meena Kothandaraman, Experience Strategist at twig+fish could be our inspiration here, she says,

Don’t be led by the method, be led by your questions.

In short, first, discuss and determine the questions your team wants to be answered, prioritize and scope the research needed, then suggest a few methods (s), discuss the pros and cons of these with your team, and based on the stage of your product development, and available resources, determine the most appropriate method.

Some questions to start with

Here are some questions to answer during the planning and before starting any study:

  • What are the questions you/your team want to answer (through this study)? This will help you understand the gaps in user research for the current as well as future studies, and guide the design of the approach for the current study.
  •  What does success look like? This will help you to get at the heart of what the study must achieve, and prioritize research questions, negotiate, or set expectations.
  •  How far is the team in the product development process? This is about what will the team do with the results and if they have enough flexibility, and resources to follow up on a study. For example, if a product is close to a release, then an exploratory study that shifts a paradigm will not be very useful for this release. Though of course it could be conducted to guide the next release.

Just a note, today’s article is a bit longer because it goes over some of the foundations of user research, so you might want to bookmark this one. It ties together the research frameworks, different types of research questions, and the methods you could use to answer some questions (or in specific research phases).

Research frameworks and how they can help structure and prioritize questions

If there are too many questions your team wants to be answered, you’d probably need some structure for prioritization. Research frameworks can help here because they help structure our questions and guide the selection of appropriate approaches.
There are several research frameworks, for example, Nielsen Norman Group and Erika Hall provide a landscape of different research methods and the kind of questions they help answer. There are also a few frameworks that present the different phases of research and related activities. These include gov.uk, the Ncredible framework by twig + fish and the research funnel by Emma Boulton. Any of these frameworks is a good starting point.

Using the NCredible framework as an example

Let’s work with the NCredible framework as an example. Once you and your team have your questions outlined, plot them within different quadrants of the Ncredible matrix (photo at the top).
There are four quadrants: Discover, Explore/Elaborate, Design, Validate. Let’s look at them one by one.

Discover

The discovery quadrant houses very open and broad questions from teams, here we are talking about even changing the basic paradigms within a field or a market. The user research here may guide the strategic directions for the company, e.g.,

  •  What is the future of transportation?
  •  What does “sharing things or services” mean within communities or neighbors?
  •  On what factors do our target groups really differ from each other?
Methods

Some commonly used methods here can be interviews, observations, diary studies and probes, ethnography, artifact analysis, etc.

Explore/Elaborate

In the elaborate phase, teams know the area that they will work on, but want to know more about the users’ needs, pains, and desires within that area. The user research here may contribute to the roadmaps of the company. For example,

  •  What are the needs, wishes, fears, and expectations of our users about topic X?
  • How do our users currently do X?
  •  What are the primary factors people consider when choosing X?
  •  What role does accessibility play in consumers’ decision to purchase/use our products? What expectations do consumers have about the role of accessibility in our company or our products? How do accessibility needs drive which information or product features our company needs to provide?
  •  On what factors do our target groups really differ from each other?
Methods and an example

Some commonly used methods here can be interviews, observations, diary studies, contextual inquiry, experience sampling, cultural probes. And different stimuli like storyboards, design probes, etc. might already start being used from here on.
Here’s an example of an exploratory/elaboration study for an e-commerce store on what people do when buying, along with their wants, needs, likes, and pain-points, etc. in the purchasing process. We used diary studies and interviews to answer these questions.

Define

In this quadrant, the company is coming up with solutions, and their questions are usually to help define the concepts further. The user research here may help determine the high-level concept direction and refine it. This may include questions like:

  •  What does the current journey for buying our latest product range on our Company Website look like? What works and what do we need to improve?
  • What information will help expert users make ABC decisions?
  •  Which concept(s) is/are the most preferred by the users and why?
  •  To what extent does our product support the user’s needs and goals?
  •  What do our users need to <insert user goal>?
  •  How do we design the navigation of our website for ease of use?
Methods and an example

Some commonly used approaches here are: co-creation workshops, concept evaluations, co-discovery, home trials, competitive analysis, card-sorting, tree testing, wizard of oz approach, etc.
Here’s an example where we created several game concepts for children and wanted to narrow down the list to a few concepts to be refined.

Validate

In the last quadrant, questions primarily concern if the users can use and understand a product well and find areas for improvement that can be fixed before release. The user research here helps fine-tune the design and/or evaluate if the product meets the UX metrics set at the beginning.
Some questions here may be,

  • Do people understand our product? Can they use it?
  • Did we meet the UX goals that we set at the beginning of the project (e.g., 80% of the users can take a photo within 5 secs.).
Methods and an example

Usability testing with prototypes of varying fidelity (moderated or unmoderated), A/B testing, data analytics, home trials, etc. are commonly used approaches here.

Also, here is an article summarizing different methods to evaluate the content and copy of the interface.

Many more methods

Only a few methods are listed above, but user research has hundreds of methods from all the fields that contribute to it (design, ethnography, engineering, HCI, human factors, psychology, marketing…) and the list grows every year.

To get inspiration on new methods, a reference book like the pocket version of The Universal Methods of Design can be very useful. It shows 100 different design and research approaches, though they don’t seem to distinguish between design method, research method, analysis, and presentation, etc.

Phew! This was all for today. Thank you for reading.

P.S. To receive these posts and event updates in your inbox weekly, you can join hundreds of researchers here: https://tinyurl.com/altuxrsign