All Articles

Articles

Cover Image

Quick & Dirty User Research
June 24, 2021

“We don’t have enough budget to spend on a research project.”

“We don’t have enough time to do research”.

If you work as a designer, product manager or researcher you’ve probably heard some of these objections to doing research. However, tight timescales and budgets are no excuses to ditch user research altogether. Specially when we all know it’s essential to make sure you deliver easy-to-use products. Quick and dirty research is a great way to get user insights fast and on a budget. Today, we want to focus on four methods that can fit into any budget — quick interviews, guerrilla testing, research workshops and design sprints.

Quick User Interviews

Interviews are a fast and easy way for getting user feedback about a topic. They allow you to learn more about your users, including their habits, needs and behaviour.

Interviews can be conducted in different places. You can ask your users to meet you in a controlled environment like a lab or a meeting room, run it remotely using videoconferencing tools or meet users at their site. Steve Portigal advices to “go where the people are”. It’s beneficial to interview people in their own environment , where they use your product and you can see how they behave.

“The information you learn when going into other people’s worlds is different from what you learn when bringing them into yours.”

Steve Portigal in Interviewing Users


Xperienz team running contextual inquiry at a workshop

User interviews can be used to:

  • Understand who are the people using your product by getting in-depth knowledge about their goals, perceptions, experiences;
  • Identify what could be designed, to inform personas, journey maps, workflow ideas;
  • Refine hypothesis about a possible solution to a problem users are facing;
  • In a contextual inquiry study, while the researcher observes users in their own environment, he asks questions to gain more insight about the context of use;
  • Guide the redesign of an existing product;
  • At the end of a usability test to collect feedback related to observed behaviours.
How to conduct a user interview
Planning the interview

Set a clear goal — As with any other research method, before conducting an interview you should clearly set the purpose of your research.

Decide who you’ll interview — You’ll want to recruit a representative sample of your target audience.

Write your interview questions — Formulate the questions you want to ask participants according to what you want to learn. That is your discussion guide, a reminder of the questions you want to ask.

Conducting the interview

Spending time with people, understanding their world and uncovering their needs is a privilege. To make the most of the precious opportunity that interviewing presents, we need to think of it as something other than an everyday chat. And conducting a good interview is hard, hard work.

Andrew Travers in Interviewing for Research

Have a partner to back you up — While you’re asking the questions, other colleague can be in charge of taking notes.

Explain the purpose — You should start by telling participants the purpose of the interview, which topics will be covered and what you’re trying to accomplish with the interview. Explain them how the data will be used afterword.

Make the interviewee feel comfortable — You can start by offering some water or a coffee and do some small talk. Starting the interview with a few ice-breaker questions like “Can you tell me a bit about yourself” will make them feel more at ease and you might also get some helpful information about the users context. You’ll want to avoid fidgeting or crossing your arms and adopt a more positive body language instead, like smiling, maintaining eye contact and nodding.

Take a semi-structured approach — Instead of asking all questions exactly how you’ve prepared them in the script (like a police interrogation) or keeping a open dialogue (like you’re just chatting in a cafe), you should aim for something in-between. Although you’ve prepared the questions and topics you want to cover, you should be flexible enough to change the order of the questions depending on the conversation flow. This will feel more natural to people and you can even get other valuable insights by asking follow-up questions.

Ask open-ended questions — You want your participants to provide detailed, expansive responses, so avoid yes/no questions. For example, if you ask “do you shop online?”, they’ll probably just respond with a “yes” leaving out any details about what kind of products they buy. However, if you ask “What type of products do you buy online?, they’ll detail what they usually purchase, and if the answer is they don’t buy online then they’ll tell you that.

Analysing interview insights

Right after the interview, take some time to organise your thoughts. Note down any stand-out moments, a particular important insight or a non-verbal cue, and discuss it with your interview partner.

Once you’ve finished all interviews, you’ll end up with a bunch of information that you need to organise in order to obtain helpful insights. Qualitative data is generally more challenging than quantitative data so you have to take some time to structure it. You can present data in a more visual and easy format, by creating mind maps, for example.

Guerrilla Testing

Guerrilla testing is a cheap and fast way to identify usability issues and get valuable insights. It consists in taking your design or prototype in front of people, whether they’re your users or not. This type of testing is carried out anywhere possible, from a cafe, to a shopping centre or a library. Participants are approached by whoever is conducting the testing sessions and asked to perform some tasks. This is one of the things that makes this method unique since participants are not recruited in advance. The sessions are normally short (around 10–15 minutes) and structured around key research objectives.

This is clearly a less formal and rigorous way of testing when comparing with a more formal usability test conducted in a lab. The truth is when used properly is better than no research at all. It’s also best to be conducted at an early stage of the design process as it’s a quick way to get feedback, identify core usability problems and help validate initial ideas.

You can use guerrilla testing to:

  • Validate how efficient a design is on its intended audience;
  • Understand if a specific functionality works how it’s supposed to;
  • Identify usability issues early in the design process;
  • Validate or invalidate early hypothesis and assumptions.
Pros of conducting guerrilla research
  • Quick and easy to run;
  • Cheap to set up and run as you don’t need expensive recording equipment or a research lab;
  • Insights can feed back the design process almost immediately;
  • Since it’s quite inexpensive, product and design teams can do guerrilla testing on a regular basis;
  • Can be conducted by anyone on the product or design team;
  • Shortcuts the recruitment process by asking directly to people in public places if they want to participate;
  • Helps to move designs more quickly through the decision process;
  • Proves to stakeholders the benefits of testing the product with real users.
Cons of conducting guerrilla research
  • You have to recruit participants, arrange location and write the test script yourself;
  • Familiarity with the project may lead to personal bias;
  • Participants might not represent your target demographic, making results less reliable;
  • As you spend a short time with participants, you’re restricted to a small part of the app or website flow.

Going out of the office and conducting guerrilla testing

Planning the test script

Well-planned and executed sessions will guarantee you more reliable insights. So before getting out there and start approaching participants, you should prepare a script for the testing session. Based on what you want to find out, create a few tasks for the participants to complete. For example, if you’re testing a mobile app for a sports store your research questions might be:

  • Can users easily search a particular sports item they would like to order?
  • Can users go through the checkout process without too much effort?

Depending on where you stand in the design process, you should prepare a more or less realistic prototype for the users to test.

It’s advisable to do a proper run through of the test before putting it in front of your participants. Testing with a friend or colleague will help you make sure that scenarios are easy to understand, that people can perform the tasks without major confusion and that the tasks fit the time you ask participants.

Finding the right participants and place

Guerrilla testing is about finding participants in any public space. However that might mean the people testing your product might not be representative of your target audience, making the results less reliable. To overcome this issue it might help to think like your audience and ask yourself where they would spend time. If you’re testing a fitness mobile app, catch people as they exit a sports store or on their way out of the gym. If you’re designing a clothes app then a shopping centre would be an ideal spot.

Wherever you decide to look for participants and conduct the testing sessions, it’s important to make sure you have a strong internet connection and that there’s not much noise.


Photo by Brooke Cagle on Unsplash

Running the test

Once you have a participant, the test should go quite similar to a normal usability test. You should ask them to think out loud as they explore your prototype, make sure they understand the test scenario and that you’re just testing the product — not them.

It’s important that you respect people’s time. If you tell them you’re only taking 15 minutes of their time, don’t push much more than that or you risk getting biased and unhelpful responses to your questions.

Although you’re short on budget you can still offer something in return to thank people for their time. Buying them coffee if you’re conducting testing in a cafe or a voucher are a good ideas.

Testing with 5 participants is enough

In the article “Why you only need to test with 5 users”, Jakob Nielsen explains how 5 users are enough to gather the necessary data. Adding more users doesn’t mean you’ll learn new things. You’ll just observe the same findings again and again. He says: “After the first study with five participants has found 85% of the usability problems, you will want to fix these problems in a redesign.”

Nielsen advocates that instead of spending all your budget conducting one elaborate study with 15 users, it’s smarter to distribute the budget across 3 small tests with 5 users each.

After testing

Now that you have some insightful feedback and uncovered user experience problems it’s time to improve your design. Analysing task completion ratio will help you understand which screens and flows of your design should be tweaked. Ideally you’ll want to tackle the most important issues first.

In Rocket Surgery Made Easy, Steve Krug advocates that you should “focus ruthlessly on a small number of the most important problems”. Instead of getting seduced into fixing the easier problems first, you should focus on the most serious ones.

Steve Krug also advises to tweak instead of redesigning. Although it might be tempting to redesign whole parts of your website or app and end up making changes that go beyond the problems observed. When fixing usability problems we should think about the smallest change we can make that we think might solve the observed problem.

Make testing a habit

Guerrilla testing can’t be a replacement for other types of UX testing. Any project will benefit from multiple research methods, be it user interviews, in-person or remote usability testing, etc.

“Testing is an iterative process. Testing isn’t something you do once. You make something, test it, fix it and test it again.”

Steve Krug in Don’t Make Me Think

Research Workshops


Photo by Leon on Unsplash

The main goal of research workshops is gathering a lot of valuable insights, in an efficient way and in a short period of time. Workshops usually achieve much more than a typical meeting and they are quite inexpensive to run. They should be highly collaborative and designed to bring together different perspectives — from stakeholders, users or both. It’s important that each workshop has a defined objective and purpose. Its structure can be quite flexible and it should be tailored to suit the project’s needs and requirements.

When running a workshop it’s important that you make participants feel comfortable with sharing their opinion and discussing different points of view. To get the conversation flowing it might be a good idea to get people out of their chairs, hand-out post-it notes and pens, and encourage them to scribble down their ideas and stick them on the wall. You can use other props and materials as well. You can use a series of games and exercises to extract and prioritise information, in a fun and more enjoyable way.

Some structured exercises to collect insights quickly:

Empathy Mapping — We can use this exercise to visualise user attitudes and behaviours, get a deeper understanding of our end users and prioritise their needs. The insights gathered will then inform the decision making process.

As we see in the NNGroup example, an empathy map is split into 4 quadrants, with the user in the middle. They are:

  • SAYS: what the user says out loud in an interview or some other usability study;
  • THINKS: what the user is thinking throughout the experience;
  • DOES: the actions the user takes;
  • FEELS: the user’s emotional state.

Affinity Diagramming — We can use this exercise in an ideation workshop to categorise and prioritise UX ideas, research findings or other relevant topics. Attendees write down each idea or fact on a sticky note. Then they analyse the notes, sort them into categories and prioritise each note. Ideally, you’ll need a big wall (or a real-time online whiteboard tool if you’re doing it remotely) where you can stick the notes, organise and rearrange them. The completed affinity diagram can then be used to inform decision making.

“Wizard of Oz” Role Play — This exercise forces participants to step out of their comfort zone. We’ve recently used this exercise in one of our workshops to understand the relationship between the users and a platform, identify its limitations and user emotions while using it.

Participants were divided into groups of two and during 5 minutes they had to recreate tasks they perform on their day to day or that they would like to perform. They used props to help visualise the task they want to perform and how they are feeling in that moment.

Props used in a collaborative workshop facilitated by Xperienz

Workshops with stakeholders

Photo by Leon on Unsplash

Normally, stakeholders have busy schedules so workshops are a great way to take advantage of the little time window they have in their agenda and get them involved in the research and design process.

The idea is bringing together people from different teams involved in the project and get distinct perspectives from across the organisation, including personal and department goals and expectations. The workshop can be used to understand the client’s needs and what’s expected from the project, or to take advantage of the client’s knowledge to draw information about the target users, typical user paths and pain points.

Workshops with users

In user workshops you invite a group of users to talk about their opinions, behaviours and feelings. Although in these sessions you rely on what people tell you, instead of on what they actually do, they can still provide valuable insights at a start of a project and help guide future research efforts.

Before the workshop, it’s important to take so time to select and create profiles of your users, to ensure there’s a right representation and you meet your desired outcomes.

A workshop ran at Xperienz

Users and stakeholders workshops

If you’re designing or redesigning a product or service it might be useful to gather both the ones that provide it and the people using it.

Sometimes what the user needs and what the organisation think they need might diverge. With the right exercises and group dynamics you can get interesting discoveries that probably you wouldn’t get from other forms of research.

This type of workshops might be useful to create a list of project requirements that truly represent what customers want.

Quick tips on how to make the most of your time

Slide from Alison Coward’s workshop at UXLx 2020

  • In a workshop on running effective workshops at UXLx 2020, Alison Coward explained that before every workshop you should ask yourself “why are you running the workshop?”, set a purpose and the expected outcomes and then prepare it accordingly.
  • In every group there’s always people with quite different personalities. While some are more willing to chat and give their opinion, others are more quite and need some time to think before sharing their thoughts. As a facilitator, you should find ways to encourage shy people to speak up and chatty people from dominating the conversation.
  • If the group is spending too long on a specific topic or going off-topic don’t be afraid to close down the discussion.
  • If you’re focused on facilitation you won’t have time to register all the data people are sharing. To make sure no insight is forgotten it might be a good idea to have a note taker with you or record the session (video or audio-only).

Design Sprints

A design sprint is a five-day process created by Google Venture designers that aims to help organisations answer critical business questions through prototyping and usability testing.

If your team doesn’t have much time to validate a project then running a development sprint is ideal for you. In just five days it’s possible to meet both the team and your users needs, prototype and validate a solution, giving your team the confidence to move forward with the project. Instead of rushing to launch a product and expected the best outcome, a design sprint helps you to validate ideas with real insights, thus reducing the risk of failure and avoiding a low ROI.

What will you get from a design sprint?

  • Quick and critical project learnings;
  • Rapid and functional prototype tested by real users;
  • Validated and customer-oriented ideas;
  • Groundbreaking and efficient products.
Running a design sprint

The Design Sprint Process

DAY 1: UNDERSTAND — At the start of the day, the team focuses entirely in understanding and mapping the problem. In the afternoon, they involve the main stakeholders in the project and decide the problem to be solved.

DAY 2: DIVERGE — The second day is exploratory, generating many ideas. In the afternoon, the team sketches them in greater detail, focusing on critical thinking. They define the recruitment profile for day 5.

DAY 3: DECIDE — The team critiques the ideas sketched the day before and decides which one will be tested. In the afternoon, they make a detailed storyboard with all the interactions.

DAY 4: PROTOTYPE — This day is dedicated to building a quick and semi-functional prototype. The team prepares the usability tests of the next day and writes the interview script.

DAY 5: VALIDATE — The last day is completely dedicated to testing the prototype with 5 users. At the end of the day, in a debriefing, the team identifies the iteration points.

Benefits of running a design sprint

Photo by Bonneval Sebastien on Unsplash

  • Identify problems: You’ll identify the challenges and needs of the project. Then you’ll map out the problem, analise it and pick an important area to focus.
  • Explore different solutions: In this fase there are no wrong answers. Through brainstorming and sketching your team explores different approaches to the problem. In the end, you select one or two to be tested.
  • Rapid and functional prototypes: You create an interactive prototype for desktop, smartphone or tablet to validate the solution. It’s not supposed to be perfect, but functional and appear real to users.
  • Test the prototypes: You’ll test the prototype with 5 users, allowing the team to learn from their feedback. In the end, you’ll discuss the results and you’ll know what’s the best path to follow.

Stats tell us:

  • The cost of fixing an error after development is 100x that of fixing it before development.
  • When organisations invest in UX during a project’s concept phase, they reduce product development cycles by 33 to 50%.

There’s no doubt time and money shouldn’t be an excuse to skip UX research during the design of a product or service. Even when funds are short and deadlines are tight every team can apply low-cost and quick methods to get valuable insights from their users and create great digital experiences.

Learn more

Don’t have much time to collect valuable insights?

Xperienz facilitates collaborative workshops with stakeholders, users or both tailored to the project goals and requirements. We can also assist you with guerrilla testing, eliminating the bias of running it yourself.

We can help you test and validate ideas rapidly with a design sprint, and understand your users needs and practices with quick interviews, both in-person or remotely.

We’ll gather the information needed to inform the (re)design of better products and services, fast and effectively.

Tell me more

Related Articles