Update: check out my book on customer development Talking to Humans
Steve Blank always liked to say, “In a startup, no facts exist inside the building, only opinions.” The lean startup movement encourages that you get out of the building with a mixture of experiments and qualitative research. Doing qualitative work gives you several benefits. It helps you learn how others experience and think about your problem space. It helps you uncover evidence about your assumptions, or lack thereof.
My post “12 tips for customer development” tries to help entrepreneurs and product designers understand how to do qualitative work more effectively. But people struggle with this area. Here are some anti-patterns to watch out for and defeat.
1. You treat speculation as confirmation
Here are some question types that I don’t like — and if you ask them, you should heavily discount the answer: “would you use this?” “would you pay for this?” “would you like this?”
I can’t say that I *never* ask these questions, but I always prefer behavioral questions over speculation.
As contrast, here is a behavior-focused interaction: “Tell me about a time when you bought airline tickets online.” “What did you enjoy about the process? What frustrated you about the process?” “What different systems or methods have you tried in past to book tickets?”
2. You lead the witness
Leading the witness is putting the answer in the interviewee’s mouth in the way you ask the question. For example: “We don’t think most people really want to book tickets online, but what do you think?” Examine both how you phrase your questions and your tone of voice. Are you steering the answer? Ask open-ended, neutral questions before you drill down: “what was that experience of buying online tickets like?”
3. You just can’t stop talking
Some entrepreneurs can’t help themselves — they are overflowing with excitement and just have to pitch pitch pitch. There is nothing wrong with trying to pre-sell your product — that is an interesting experiment unto itself — but you should not mix this in with behavioral learning.
If you do try to pre-sell, don’t just ask “would you pay for this?” but rather ask them to actually pay, and see what happens. Some people ask the question, “how much would you pay for this?” but I do not. Instead, try actually selling at different price points. I much prefer having the potential customer experience something, rather than speculate over something.
4. You only hear what you want to hear
I see some people go into interviews with strong beliefs about what they like and dislike. When you debrief after their custdev conversation, it is magical how everything they heard aligns perfectly with their opinions. Our brains are amazing filters. Leave your agenda at the door before starting a custdev conversation. One way to solve this is to have two people for each interview — one person to ask questions, and the other to take notes.
5. You treat a single conversation as ultimate truth
You’ve just spoken to a potential customer and they have really strong opinions. One instinct is to jump to conclusions and rush to make changes. Instead, you need to be patient. There is no definitive answer for how many similar answers equals the truth. Look for patterns and use your judgement. A clear, consistent pattern at even 5 or 10 people is a signal.
6. Fear of rejection wins out
This is one of the biggest blockers to people doing qualitative research, in my experience. Both fear of a stranger rejecting your advance or rejecting your idea. Many excuses, such as “I don’t know how to find people to talk to”, are rooted in this fear. JFDI. Customer development isn’t just about street intercepts. You can recruit people on Craigslist, Facebook and LinkedIn groups, and good old fashioned networking.
7. You talk to anyone with a pulse
I see some teams taking a shotgun approach. Instead, define your assumptions around who your customer will be, and who your early adopter will be. You might even do a lightweight persona (see the book Lean UX for examples). Zoom in on those people and try to validate or invalidate your assumptions about your customers. It is ok to occasionally go outside your target zone for learning, but don’t boil the ocean. Focus, learn, and pivot if necessary.
8. You wing the conversation
If you go into a custdev conversation sloppy, that is how you will be perceived. Instead, write up your questions ahead of time and force-rank them based on the risks and assumptions you are worried about.
To define your assumptions, you can answer the questions we ask in www.neoincubate.com, do a business model canvas, or tackle the product points Marty Cagan lays out. Your exact method doesn’t matter as much as the actual act of prioritizing your risk areas.
During your actual interview, do not literally read your questions from a piece of paper, but rather keep things conversational (remember, you are getting the subject to tell you stories). If you uncover something interesting, follow your nose and don’t be afrad to diverge from your initial priorities.
9. You try to learn everything in one sitting
Rather than trying to go as broad as possible in every conversation, you are actually better off zooming in on a few areas which are critical to your business. If you have a huge range of questions, do more interviews and split the questions.
10. Only the designer does qualitative research
It is ok to divide and conquer most of the time, but everyone on the team should be forced to get out and talk to real people. Note: you will probably have to coach newcomers on #5’s point about not jumping to conclusions.
11. You did customer development your first week, but haven’t felt a need to do it since
It is always sad to see product teams start things off with customer development, and then completely stop once they get going. It *is* perfectly fine to let custdev work ebb and flow. If your learning curve flattens, it can make sense to press pause on custdev or change up how you are doing custdev. However, you want to build a regular qualitative cadence into your product process. It will provide a necessary complement to your quantitative metrics, because it will help you understand the reasons why things are happening.
Bonus 12 (added). You ask the customer to design your product for you
There’s a famous line attributed to Henry Ford, “If I had asked people what they wanted, they would have said faster horses.” Remember, it is not the customer’s job to design the solution. It is *your* job. It is the customer’s job to tell you if your solution sucks. Get feedback, yes. Remember that the further away you are from a working product, the more you have to filter what you hear through your judgement and vision.
p.s. as with all tips on lean and agile, there are always places and times to break the rules and do what is right for your context, and your business.
p.p.s. if you want to learn more about lean product design practices, check out the book Lean UX.