Case Study: Customer Interview Questions

Have you ever gone out on a first date, had a really fantastic time, then ended the night by asking your date “So… do you love me?”

Yeah. Neither have I. Because that’s a terrible question.

As it turns out, your grade school teacher lied to you. There is such a thing as a bad question.

So when you’re doing customer development and talking to customers, how do you know if the questions you’re asking are the business equivalent of “Do you love me?”

A few days ago, an online education startup asked if we’d give them some feedback on their interview questions. After giving them feedback, they gave me permission to share the before and after’s with you.

Word of clarification: In the early stages of customer discovery and customer development, do not send out surveys to collect data. This company had someone doing the interview, but used the survey afterwards to input the information into a central place.

There are pros and cons to this system – the biggest con is that it’s difficult to record what you learn from follow up questions and stories they tell. Surveys, even if the interviewer is only filling it out afterward, encourage you to keep drilling through the survey and not dive deeper.

We’re going to list their original question, and then our comments and suggestions on that question. If you want to see more examples, you should watch our Whiteboard Wednesday video on Good and Bad Examples of Customer Interviews.

Bad Question #1: Age Ranges

Bad Question #1: Age Ranges

Why do you break out their age into radio buttons instead of a text field? In doing this, you lose a lot of resolution in your data. At 25, you’re freshly out of college, trying to figure out your career, and still trying to find your way in life. At 34, you’re over 10 years out of college, might be in the middle of a career, and might have a family and children. You’re in a very different place than at 25 than you are at 34. Careful about bracketing ages.

Bad Customer Interview Questions

Bad Question #2: Employment

There should be check boxes here. I might be employed and also a student. Or I might be a student, but also looking for a job. Or employed and looking for a job! Allow for this flexibility in responses.

Bad Customer Interview questions

Bad Questions #3: Do you need to learn new skills?

This is a borderline ice cream question. Almost no one would say “No, I have all the skills I will ever need.” So what’s the point in asking it?

This question also doesn’t necessitate an answer beyond “yes” or “no”. What do you learn from someone answering “yes” to this question? By rephrasing it to “What did you struggle with at work today?” or “What was the last thing you struggled with at work,” you learn about a particular problem and can dive deeper into what skills would help. Look for stories, not statements.

Bad Question #4: What do you need to learn?

Bad Question #4: What do you need to learn?

You’re asking about their future self, which means you’re likely to hear about their ideal self (instead of their actual self).

“Do you want to go to the gym and work out more?” “Of course!” Does anyone ever actually start going to to the gym more? Rarely. (Kudos to the three of you who do!) Remember, past behavior is the best predictor of future behavior.

A better line of questioning might be “Tell me about a time in the last three months where you needed to do something you didn’t know how to do.” “How did you accomplish that task?” “What new skills are you learning right now?” “Why?”

By learning what they’re doing NOW and how they’re solving their problem now, you can learn about what skills they ACTUALLY need and want to learn.

Bad Question #5: Have you recently used any online learning tools?

Bad Question #5: Have you recently used any online learning tools?

This is the right idea, but you’re pushing online learning tools on them. You want to pull it out of them. If they mention online learning without being prompted, then dive into it.

If they DON’T mention online learning… then that’s very interesting. Did they ever consider it? Is there no online course for their desired skill? Did they have a bad experience with online learning in the past? You’re not going to get the same purity of data if you directly ask them “Have you used online learning before, yes/no?”

Bad Question #6: What is the quality of the current offerings?

Bad Question #6: What is the quality of the current offerings?

Instead try: “Tell me about the tools you’ve used in the last 3 months.” “How was the experience of using it?” Get stories from them. If the tools don’t meet their needs, dear god will they tell you about it.

Bad Question #7: Describe critical features that don't exist today

Bad Question #7: Describe critical features that don’t exist today

Be very careful asking customers what they want. They’ll typically send you chasing unicorns. “But what I REALLY want is a car that can hover and uses sadness as fuel and produces happiness as exhaust.”

These types of questions should be less suggestive/leading. Get them telling stories. Why do they want a car that can hover? They can never find parking outside their office? Interesting. If there are features that they need or would like, it’ll come up naturally in conversation. Don’t expect the customer to tell you what to build – just rely on them to learn more about the problem.

Also, remember that people often have no idea what they want or what actually works well for them. They may say that “the videos are the best part!” when really the video is just first to mind; really the newsletter is what keeps them coming back.

Bad Question #8: Are you willing to pay?

Bad Question #8: Are you willing to pay?

Ice cream question. “I have a lotion here that will solve all your problems. It will regrow your hair, make you three inches taller, and make you vomit Benjamin Franklins. Would you pay for it?”

Who would SAY no to a product that you tell them solves all their problems? Instead, look at what they ALREADY are paying for. Past behavior is the best predictor of future behavior. Have they ever spent any money on online learning? Self education books? Night school? THAT is a good indicator.

Bad Question #9: How much would you pay?

Bad Question #9: How much would you pay?

They’re going to lowball you. Also, they have no idea. Instead, infer what they’d pay based on what they’re already paying for. Or try to get them to put a dollar amount on the value it would add to their life (could they make an additional $5k/year with their new job? $20k/year? Now you have something to base your price off of.

So remember – there is such a thing as bad questions. Don’t ask someone on the first date if they love you, and be careful what questions you ask your customers.

Enjoy this article? Sign up for our mailing list and get an email every Wednesday with links to new videos and articles about Customer Development, Lean Startup, and more!

3 thoughts on “Case Study: Customer Interview Questions

  1. Pingback: Case Study: Customer Interview Questions | Rocketboom
  2. Pingback: "There is such a thing as a bad question" Thanks Zach! | Jason GrimesJason Grimes
  3. One thing that I had wished we included in the survey we did was geographic location. Even if you ask great questions, if the people who take your survey aren’t in your target audience (for us that was US only) then it can skew your data and the results.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s