No, You Don’t Have to Be A UX Researcher to Conduct a (Good) User Interview

Small startup with no UX Researcher on staff? Here’s why user interviews are worth doing anyway – and tips on how to get started.

Spend enough time in startup land and you’ll stumble upon one of the holy commandments of SaaS (software as a service) businesses: Thou Shalt Talk to Your Users Early and Often.

That means not just doing usability testing (putting a mockup or a new feature in front of users to see how they interact with it), but also picking up the phone to learn about users and potential customers before you build, mock, or even brainstorm.

Unfortunately, implementing a consistent user research practice is like building a regular fitness routine: we all know we should do it because it’ll be good for our short- and long-term (business) health, it’ll give us iterative and actionable feedback that will help us improve more quickly, and it has a tendency to spread positive ripple effects to other areas of our lives (and our companies).

And yet, too many startups do the equivalent of driving past the gym every morning by putting user research on the back burner for an ever-distant “someday.” If you’re at a small startup like ours, the justification is often a lack of resources — with a tiny team (that already wears multiple hats) and limited capital (that takes a lot of sweat equity to raise), user research can understandably feel like a “nice to have” luxury, one meant to be saved for the day when your company finally makes it and can afford to hire a snazzy UX researcher.

Well, we’re here to channel our best Richard Simmons-esque level of encouragement and let you know that the benefits of asking your customers for feedback are well worth the work, even and especially if you’re on a team that lacks a dedicated UX researcher. Here’s how we’ve approached user interviews at Magoosh from the very start, how they’ve helped our business grow, and tips and advice for getting started.

Three benefits we’ve gained from conducting user interviews with our students

Image credit: Thinglass (licensed via Shutterstock)

We’ve conducted user interviews in some iteration for nearly every quarter over the past ten years. There are three main motivations—and resulting benefits—behind our prolific user research habit:

Reason/Benefit #1 – Offer Better Products

As a relatively young startup with a global student population, achieving product-market fit has been crucial to our growth and success.

It’s allowed us to make it to profitability on just one seed round of funding, grow our product line from our staple GRE offering to products helping students prep for over nine standardized tests, and invest in expanding our team, fostering great company culture along the way.

Throughout our now decade-long journey, our growth has been catalyzed by product feedback from our users, from our recent addition of text-message reminders to our ACT and SAT products, to our 2010 decision to switch from user-generated to expert-written content. Today, though our products can always continue to be improved, we’re stoked to see our net promoter (NPS) scores overall on the rise, and our student numbers continuing to climb from eight million served.

Reason/Benefit #2 – Check Our Assumptions

We’ll be the first to admit that, even amidst the fiercely proud-nerd culture of Magoosh, there’s still SO much we don’t know!

When we first entered the SAT market, we assumed parents would be the louder voices we’d need to listen to. A round of user interviews later, we realized the story was actually much more nuanced: many students were studying on their own, with little or no parent involvement.

It’s in the spirit of one of our core company values, Learning > Knowing, that we acknowledge our blind spots and seek to listen to our students in the hopes of learning what they can teach us or broaden our perspective around.

Reason/Benefit #3 – Deepen Empathy For Our Customers

The world of education is rapidly changing, and even as lifelong learners, it can be incredibly hard to keep up.

This is especially true as the college admissions landscape becomes more complex and less accessible to students and, in line with our mission, user interviews have afforded us a personal, humanizing means of empathizing with our customers and their wildly diverse lived experiences. We would never have known what it’s like to prepare for the TOEFL in Iran if we didn’t talk with people doing exactly that.

When – and how – to start conducting user interviews

Image credit: Notto Yeez (licensed via Shutterstock)

When should you conduct user interviews?

It’s important to learn to differentiate between when a user interview is the best path for learning, and when it might actually be a distraction. Generally, user interviews are better for focused learning over open exploration, such as talking through challenges and problems with your users. The more specific the info you’re trying to get, the better a user interview can help you inform next steps.

Some points to consider:

Should it be a user call?
Whether you are trying to identify a problem, get more information about a known problem, or see if a solution will work, you should consider whether user calls are the best tool to get what you’re looking for. Does the information you want already exist somewhere, or are there better methods (like a survey) to collect it? User calls may or may not be the route you want to go.

Define your purpose
Are you conducting “discovery calls” to try and understand problem spaces, or have you defined a problem space and want to do a deeper dive into how users may be solving this problem already? This will additionally inform who you reach out to and the questions you ask.

What does success look like?
User interviews are a means, not an end. If the sum of your user calls leads you to the next stage of your work, and that work is successful, then the calls were a success.

How to recruit (the right) participants for user interviews

Image credit: Ollyy (licensed via Shutterstock)

So you’ve gotten clear on the information you’re trying to learn and decided that user interviews are indeed the best route. Now, who in your customer pool should you reach out to?

Who you recruit for a user interview depends on what questions you are trying to answer
Recruit users who will provide the “best” source of information for the problem you are tackling. Take into account the various stages of your customer journey (for example, users already aware of your brand who have yet to purchase? Trial users? Power users? Users whose engagement has dropped off recently? Blog readers who never engage with your product?) to inform what subset of your users makes the most sense.
Match your research group proportionately to your customer population
Working on your core product? Well, if 70% of your customers are domestic and about 30% are from India, then aim to recruit a cohort of users that matches this 70/30 breakdown. If you can’t get there, do more recruitment, and don’t hesitate to turn some volunteers from over-represented groups down when you need to (kindly, of course!). Self-selection is a real thing that can skew the makeup of your pool of users so you’ll need to find ways to target your outreach so that your research is proportionate to your user base.
Plan a funnel to determine how many users to recruit
Start with how many users you ultimately want to talk to. Next, estimate the percent of users who you think will sign up and not show. Then, the percent of users who will open your email and percent who click through to sign up. Play around with this funnel (using historical data on the sign up and open rates of your customers, if available) until you understand how much outreach you’ll need to reach your target number of interviewees.

Note: this initial outreach number might be bigger than your pool of potential candidates. If that’s the case, you may need to sweeten the deal a bit with (bigger) incentives or build out a more robust campaign to increase open and sign-up rates.

Talk to as many users as you have time for, prioritizing the recent customers
To find useful themes, you’re going to need to talk to more than 5 customers. We’ve found 10 works as a minimum for a lot of purposes, but shorter, less dense calls may warrant 20 or 30. For most interviews, recency matters. Aim to recruit active customers from the past few weeks as opposed to the past few years. They are in the midst of using your product and not trying to remember what it was like. Oh, and if you happen to be talking with low-intent or low-engagement users, get them on the phone immediately. The longer you wait after booking, the more likely it is that they won’t show up for the call.

Tips and tools to streamline your user interview process

We’ve made a lot of mistakes conducting user interviews over the past 5 years (and discovered a lot of useful habits along the way!). Here are seven tips and tricks:

Create a script and have someone review it
Having a script will help you avoid leading questions, control your time, and stay focused on your goals. But be prepared to shuffle the order of your questions and ask follow up questions.
Find a notetaker and tell them what type of notes to take
Humans are actually really bad multitaskers. Understandably, it’s hard to have a high-level picture of a conversation when you are in the middle of it. To make the most of each of your user interviews, it’s best to have one person focused on asking questions, listening, and responding while someone else primarily takes notes.Be clear about the type of notes each person should take—verbatim, paraphrasing, or highlights. When you have two people, you can take two different approaches. It’s inevitable that you’ll forget some details, so if you have one person taking notes as close to verbatim as they can get, you’ll be able to return to exact phrasing after the fact, which will help prevent false memories. But if there’s only one person in the room, conducting the interview, taking verbatim notes, and seeing the big picture becomes impossible.
Spreadsheet tracker, not a bad idea!
With multiple calls, and potentially multiple people making calls, having a spreadsheet with the users, relevant info, and recaps from the user calls is a great way to stay organized—and know who needs to have a gift card or other reward sent to them.
Ask users for multiple modes of contact
Technical difficulties WILL happen, and it’s helpful to plan for them in advance. When recruiting users, ask for their phone number (or other additional contact method) so you can call them instead of having them call you.
Schedule smart, schedule often, and send reminders
Schedule less engaged users as soon as possible in your interview window to mitigate against no-shows. In that same vein, overbook more users than you ultimately need to talk to because people will flake. To account for the faulty nature of human memories, text reminders to users an hour before the scheduled call to verify they’re still available.
Plan for a debrief + aim for 2-4 takeaways
Ideally, you discuss the call immediately after the call, so make sure to reserve time in your schedule before the next one. Discuss what stood out and arrive at 2-4 takeaways. If you’re pulling out more than 4 highlights, they’re probably not equally important—try to keep it to only the meaty points so you can focus on what really counts.
Know thyself
Some people can rock through eight user calls in a day. Others may find it challenging to do more than a few in a day. Plan the interviews so that you don’t burn out.

Troubleshooting common user interview challenges

Image credit: Juergen Priewe (licensed via Shutterstock)

You’ve checked off all the right things (defined a clear purpose and outcome, recruited the best users from your customers, set up a solid process and roles) but find that your questions are falling flat during the interview. What gives?

Not convinced you’re framing the right questions during your user interviews? Avoid these common mistakes.

Avoid asking leading questions
Asking leading questions is the best way to invalidate a response or even an entire call. If you ask a leading question (even one!), you may have to throw out the entire interview, so practice as much as you can to not ask leading questions. No yes/no questions. Ask who, what, where, when, why, and how questions. “Can you tell me about x?” and “Can you tell me more about y?” are good! Also, practice might just mean conducting interviews where you make mistakes and the results are thrown out, assuming that you use those learnings to improve future interviews.

Priming is not the same as leading
It can be helpful to start broad and narrow into the questions you want to ask. Start with something concrete and that will make it easier for them to answer questions later. Instead of “How did you use Magoosh to study for the SAT?”, start with, “Tell me about the last time you studied a new topic.”

Take answers to quantitative questions with a grain of salt
People tend to search for the “right” answer or provide an answer that you want to hear. So you won’t get an honest response if you are asking for something on a scale. The NPS question (“How likely are you to refer…”) when asked on a phone call is likely going to result in a higher score than the same question asked in another medium—your interviewees will try to make you happy, and may not give the most accurate response. These kinds of questions can be helpful to lead into follow-up questions, but they’re dangerous when used on their own.

Covered all of your bases? Here are a few bonus tips to be double sure you’re not leaving insights (and honesty) on the table during your user interviews.

You won’t hurt my feelings
Start the call with why you are conducting the calls. And tell them you want them to be brutally honest, and that they can’t say anything to offend you. People will unconsciously provide answers that they think you want to hear or that they think true of themselves. Prime them with a statement at the beginning that hopefully dampens this.

I know what you did last summer (or last week)
Similar to setting the stage, let the user know you have their usage history in front of you, if you do. Starting with a quick “I see you spend a few hours using [our product] each week”—or whatever happens to be true for the particular interviewee—can help avoid situations where people claim to have done much more or much less than they actually have. Not everybody wants to admit how much time they spend on an entertainment service, and many of us will claim to have used that exercise app as much as we wished we had, not how much we really did.

Silence is golden
Sometimes the best way to get someone to talk is with silence. Let there be a space so people can talk, and think, and talk some more. The impulse to respond right away can be a difficult one to fight, but there’s very often something more that will come out unprompted (with no danger of having led them to it, to boot!)

Ready? Set? Research!

Image credit: Yummyphotos (licensed via Shutterstock)

Starting to conduct user interviews (and doing them well) can seem like a lot of hassle and upfront effort, especially when you’re working with limited resources, but the barrier to entry is not as high as you might think. They’re also a tremendous amount of fun! For us here at Magoosh, we most appreciate them as a way to connect with and build even greater empathy towards our students.

Even if you’re not ready to lead a user interview project yourself, we’d highly recommend you volunteer to be a notetaker for someone else at your company conducting user interviews, as it’ll help you be better prepared when your time comes.

We hope you find these suggestions helpful, and welcome any thoughts or questions in the comments below!

If you’d like even more resources on user research, here are a few of our favorites:

Author

Hannah Baker

Hannah helps build and manage the team of expert remote tutors who tackle all of students’ toughest questions. She has both B.A. and M.A. degrees in Chinese from Williams College and San Francisco State University, respectively, and has worked in test prep since 2010. She can rarely be found at home, but often on a rock face, singletrack, or mountain top.

View all posts

0 Shares