Difficult Participants in remote user interviews

Photo by cottonbro studio: https://www.pexels.com/photo/white-red-and-green-floral-textile-4614249/

No-shows, late arrivals, distracted environments and participants with some extreme behaviors in remote user interviews are painful scenarios for researchers and participant recruitment companies alike. Despite best efforts to avoid unsuitable participants, slip outs still happen and can at best only be minimized (not eliminated) with due diligence, well-defined SOPs at the recruitment stage. To see such participants as an opportunity can be actually rewarding in many ways.

Beyond the researchers themselves, a mishandled interview session with such participants could be disappointing for stakeholders / Observers, Note-takers and recruitment partners because specific user profiles can cause distraction. 

While people are generally nice and are willing to help by giving their feedback, there are participants who could make a user interview totally ineffective. Although there are ways to handle such participants, every user researcher is not trained to handle these situations and still get valuable insights from the session. Especially when the interview mode is over a video call using remote user interview tools or video conferencing tools e.g. Zoom, communication is one level tougher to handle as compared to in-person interviews. 

In the rest of this article, we discuss how researchers could manage such typical participants to extract insights and avoid skewing of insights. There is some learning for the new researchers in these tips so that they can work around when interviewing such participants. Expecting a participant to behave the way you as a “Researcher” expects them to is honestly, an unrealistic ask. “Firing” the participant may feel convenient even if it is not completely optimum or fair either. 

Afterall, some of those people would also use your product, so learning to tackle specific user behavior in your user interviews isn't all fruitless. Learning this makes one a better researcher too since these situations need to be experienced and tackled instead of asking ChatGPT or reading in books.

Users don't behave in a certain way on purpose. “Let me not be helpful to the research”, thought no user, never! 

We have put together a list of user types to watch out for and the likely way out in case one is interviewing for such a profile. Instructing the recruitment company would not be sufficient to net out all those participant types one didn’t expect.

Happy & Chatty

One of the easiest to slip through the recruitment process, these people are generally fond of chatting and sharing as much as they can. Often easily mis-perceived “articulate” over phone screening, their irrelevant elaborate responses can completely off track the researcher during remote interviews.

Set the rules, Clarify the question.

in the beginning of the interview, clearly mention to the participant 

“As a part of my role I need to carefully make the most of the time I have with you, I’d cut you off at some points! Please be aware of that” 

As soon as the participant responds with too much side information than needed, interrupt and repeat your question assertively to clarify what you are looking for. This must be done in the first instance itself. 

Lets’ say a researcher asks a question “Why did you choose this app A over B?” This could get the participant of this type to start and go on forever like this:

“Since the beginning when I started trading about 6 and a half years ago, I found that my 4 year old Dell laptop is too bulky to carry along. The ones with a 15.3 inch screen, you know? I’m on the move almost the whole day. My friends suggested purchasing a tablet but that’s such a difficulty to type on. Carrying around is ok, specially ok for real-estate and insurance agents, not for me. The iPad looks better and elegant too. On-the-go I used to like tablet earlier until the time I went on a business trip to Japan the battery used to drain off too quickly…….blah de blah de blah” 

The participant is perhaps coming to the App A vs B topic at all, however they have gone on a long detour. This is exactly the time to interrupt and re-establish the rules of the user interview. While being nice and respectful, the researcher must say like this:

“I’d like to pause you there! To be clear, I am trying to understand your reasoning of using App A over App B. I need you to respond with specific reasons. I’d ask you follow up questions if I need additional information”. 

After this, the researcher may repeat the question by saying: 

“So, can you please let me know two or three reasons of using App A and not App B?”

This would generally be sufficient for the participant to recall that s/he did not answer what the researcher was looking for. It is possible that this participant has never attended a user interview earlier, so they mistook it for an informal video call session and their usual instinct / habits took over.

A researcher must not feel odd to interrupt and clarify as many times as needed. The time with participants is super precious and it is the researcher's job to make the most of it. Remember to be assertive in a remote research interview even if it feels repeating oneself more than one time. Remember that the participant can’t make much of your physical gestures and body language as much as they could during an in-person interview.

Pessimists & Optimists

Different people’s thought engines are built differently and over time as they experience life. 

In your research you may encounter people who completely reject your ideas or are cynical about the possibility for your idea or solution to exist or be useful. As a personality trait, such participants might respond with extreme answers and use words like “hate”, “never” “impossible” and may even exhibit strong emotions as they respond to questions. 

Another participant could be at an optimistic extreme. For these people, any or everything would just work and every question is supposed to be met with an unrealistically positive response. The issue with those answers is that might not really be how things would work in real life for them.

For either of these personalities, the researcher must be supportive and neutral. Giving them real life scenarios along with clear expectations about what answers you are seeking is helpful when interviewing such users. 

For instance when asked "The job search posed some difficulties to you, How did you manage those?” the response from an over optimistic user could be:

“Not at all! Overall I'm feeling really positive and excited about my job search. I've had some great interviews recently, so I'm fully confident that the right job is somewhere pretty soon on the way. For some people it can be tough, but I'm always open to new opportunities no matter how the environment is."

The same question to a naysayer would result in the following likely response:

“Super difficult! This job market is terrible, and I'm competing with many others for every job opportunity. Employers are so picky these days, and they're not willing to give a chance for a second interview to people like me. I already made several applications and didn't hear back from anyone. What a waste of time”

Both the responses above are not constructive at all.

They share some perspective, which is largely personal. A constructive response could contain what specific difficulties they have been facing, what efforts have they taken to overcome the situation, what has worked and what hasn’t.

To handle such participants whether over optimists or cynics, the researcher needs to be detailed and specific in their questions, remain neutral and not show any empathy to the responses. 

The researcher could also try asking questions in a different way. One of the ways that always works for the researchers while dealing with naysayers / over-optimistics is to break down of the question into multiple smaller, timeboxed and explicit questions. For instance, 

“What have you done in the last 2 months to handle the difficulties in job search?” 

That question could result in a more directed response instead of a point of view. Detecting such a profile of participants is especially hard for researchers at the outset since expertise in understanding facial expressions is an art in itself.

Designer & Researchers

It is not always bad to interview someone who is a researcher or a designer. The users who are from this role need to be managed differently during a remote user interview. A good sign you have unintentionally invited a designer or researcher for interviews could be when you hear one of the following or similar statements:

“I can completely see why this design won’t see the light of the day.“

“This is so beautiful, much better than a similar one I once did.”

“The colors are way off, a local designer would have designed this better?”

“This design will not work with most users”

“I understand what you are asking, this question can actually have multiple answers.” 

“I always found focus groups to be more effective than interviews”

Researchers must go deeper into the answers to understand the root cause. Reaffirming the responses and encouraging the design and research inclined statements from the participant would result in more of such responses. 

The researcher needs to clearly tell the participant what exactly you are not looking for in this research.

Reframing the question and bringing focus back to the question helps here. Redirecting the participant from the aesthetic usability aspects to the navigation aspect can be useful. An advantage that the researcher has on their side is the ability to use specific terminology which normal users would not be able to understand. Asking them to be  “constructive”, “think aloud” or terms like “color theory” would have a better chance of eliciting the desired response.

In the remote user interview setting, a researcher can also ask the participant to share the screen and show some non-confidential research and design work they have done. By spending a few minutes on this topic, the researcher can first find out the topics and suitability of such a participant for the interview. From here, the researcher can determine whether they want to continue with the candidate or is it a total misfit.

Busy & Silent

Especially in remote user interviews, such participants can be either busy, very short in their answers or both. The signs are obvious:

Short one word responses e,g, “Yes” “OK, yes” “No” “Hardly”, “Often”

Eye focus not towards the screen / camera 

People moving around in the background

Moving background scenery during the interview

Shaking camera

Video call drop offs

Participant joining via mobile device even if asked to join via Computer

Need to repeat the question every time

Question misunderstood

It would be sensible for researchers to tell the participant as to why their responses are not helping the research. It could also just be that the participant is shy or hesitant to express themselves due to personal nature, language proficiency or cultural issues. Depending upon the situation, the researcher can decide to pause or stop the interview, re-orient the participant and if those don’t work then, continue at a later point in time.

With the reopening of workplaces, many people are not working remotely anymore. Therefore scheduling your interviews at timings when people are more likely to be off work makes a lot more sense. This would definitely require the researcher to be more flexible and work around their schedule but it's worth the sweat. The stakeholders would also appreciate the efforts the researchers put in their work if the reasons are shared with sufficient background. Nothing beats the return on high quality user insights, does it?

Interviewing a variety of participants is always helpful. It might seem hard to deal with extreme participants, although it is a blessing to gain experience and a more nuanced understanding of users' needs. That can help in building better user Personas and a deeper level of understanding about your product usage. Researchers who learn to deal with such participants end up becoming better at their craft. They develop more empathy to inform the business and build better products. So next time you meet that imperfect profile, see it as an opportunity!