Note: also see my 11 Customer Development Anti-Patterns post. Each time I give a talk introducing people to qualitative “customer development” conversations, I try to revisit my points. A few months ago, I gave this talk to an entrepreneurship class at Columbia Business School, and once again the list and messages evolved. Below you can find my latest thinking (here is version 1 and 2)
1. One person at a time
Focus groups are a group-think, distraction-filled mess. Avoid them and only talk to one person at a time. If desired, you can bring someone with you to take notes — some UX designers like this approach. Personally, I tend to do one-on-one interviews because I think people loosen up and thus open up a bit more, but it can be nice to have a note-taker, which allows you to focus entirely on the conversation and body language.
2. Know your goals and questions ahead of time
Have your assumptions and thus learning goals prioritized ahead of time. Decide who you want to talk to (age, gender, location, profession/industry, affluence, etc), and target interviewees accordingly. Prep your basic flow and list of questions. You might veer off the plan to follow your nose, which is great, but go in prepared.
3. Separate behavior and feedback in discussion
Decide up front if your focus is going to be on learning a user’s behavior and mindset, and/or getting direct feedback or usability insights on a product or mockup. Do not mix the two in the discussion flow or things will get distorted.
Put “behavior and mindset” first in your discussion flow. During this part, don’t let the interviewee go too deep in terms of suggesting features, but keep them focused on if they have a problem, how they think about the problem space, and if and how they have tried to solve it in past.
If you want to get feedback on a product, whether on paper or digital, do this after digging into behavior and mindset.
4. Get psyched to hear things you don’t want to hear
If you don’t do this, you might find yourself selling or convincing, or even hearing what you want to hear. This is called “confirmation bias” and we are all very susceptible to it. Your initial goal should be learning.
You might, however, run an experiment where you test the market, test pricing, and do try to close a sale. That is great, but keep this part of the conversation separate. As with product feedback, keep this out of the behavior/mindset portion of the discussion.
5. Disarm “politeness” training
People are trained not to call your baby ugly. You need to make them feel safe to do this. Ask them up-front to be brutally honest, and that this is the very best way they can help you. If they seem confused, explain that the worst thing that could happen is to build something people didn’t care about.
6. Ask open ended questions
Do not ask too many yes/no questions. For example, minimize such questions as “do you like Groupon?” Instead ask “what kinds of deals do you look for, if any?” “What motivates you to hunt for deals?” “How do you discover deals?” “Do you get frustrated with the deal sites out there?”
Sometimes it is hard not to ask a yes/no question, but always follow up with an open-ended question like “why?” or “tell me more about that experience.”
7. Focus on actual behavior, not speculative or abstract feelings
To emphasize #3: people are not very good at predicting their actions, knowing what they want, or knowing their true goals. Your job is not to ask the person for the solution. It is *your* job to figure out the best solution, and then validate that your solution is actually right.
People *love* to talk about features and solutions. When you are in learning mode, don’t let that dominate the conversation. Try to keep things factual. Get them to tell you stories about how they previously experienced a problem, if they tried to solve it (or why not), and what happened. Get them to tell you stories about using other products that are in the same domain space. You do want to dive into their emotions, but you can trust a discussion of historical emotions much more than one speculating “what ifs”.
Some people like to ask the question, “if you could wave a magic wand and make this product do whatever you want, what would it do.” Occasionally interesting things can come from this, but I would advise that you take the answers with a grain of salt.
8. Listen, don’t talk
Try to shut up as much as possible. Try to keep your questions short and unbiased (i.e. don’t embed the answer you want to hear into the question).
Don’t rush to fill the “space” when the customer pauses, because they might be thinking or have more to say.
Make sure you are learning, not selling! (at least not until that part of the conversation, if relevant)
9. Follow your nose and drill down
Anytime something tweaks your antenna, drill down with follow up questions. Don’t be afraid to ask for clarifications and the “why” behind the “what”. You can even try drilling into multiple layers of “why” (see “Five Whys”), as long as the interviewee doesn’t start getting annoyed.
10. Parrot back or misrepresent to confirm
For important topics, try repeating back what the person said. You can occasionally get one of two interesting results through this. In the first, they correct you because you’ve misinterpreted what they said. In the second, by hearing their own thoughts, they’ll actually realize that their true opinion is slightly different, and they will give you a second, more sophisticated answer.
Another approach is to purposefully misrepresent what they just said when you parrot it back, and then see if they correct you. But use this technique sparingly, if at all.
11. Ask for introductions
At the end of every interview, see if you can get leads to another 1 to 3 people to talk to.
If it is not obvious to everyone by now, let me just be clear that you want to avoid doing these interviews with friends and family. There are lots of creative ways to recruit interviewees (the tactics vary depending on who you need to get to), but getting referrals will make the process a lot easier.
12. Write up your notes as quickly as possible
The details behind a conversation fade fast, so if you haven’t recorded the session, write up your notes and color commentary as soon as you can. I brain-dump into a shared Google Doc so the rest of the team can see it. (Note: I typically have not recorded sessions for fear of making interviewees more self-conscious or careful, but other entrepreneurs have said to me that, while it takes some rapport-building at the start, pretty soon people forget about a recorder.)
Afterwards: Look for patterns and apply judgement
Customer development interviews will not give you statistically significant data, but they will give you insights based on patterns. They can be very tricky to interpret, because what people say is not always what they do.
You need to use your judgement to read between the lines, to read body language, to try to understand context and agendas, and to filter out biases based on the types of people in your pool of interviewees. But it is exactly the ability to use human judgement based on human connections that make interviews so much more useful than surveys.
Ultimately, you are better off moving fast and making decisions from credible patterns than dithering about in analysis paralysis.