This post originally appeared on Medium but I am now posting more regularly here.
Before you rally a team, spend money, and move heaven and earth to bring a new product or new company into being, wouldn’t you want to de-risk your vision, timing and execution? Wouldn’t you want to make more informed decisions? There are two great places to start: talking to your potential customers and running experiments.
In 2014, I wrote the book Talking to Humans, with the collaboration of Frank Rimalovski, to tackle the first of those two. That book ended up being a worldwide hit, but we knew we didn’t want to stop there. We finally released the sequel Testing with Humans to cover the experiment side of things.
The journey to the first book began with a list of 12 tips for customer development. So it was always inevitable that the topic of experiments was going to get the same treatment. Our book goes into a lot more detail, but for those of you who love breaking down the essentials, here you are:
12 Tips for Running Effective Experiments
- Save your experiment effort for risks that will truly impact the success or failure of your project or business.
- Don’t just think about experiments for your product. Remember to examine your customer segments, value propositions, customer acquisition methods, pricing plans and revenue models, unit economics, etc.
- Stretch your thinking at the start because there are always more ways to test something than you think.
- Be disciplined about the details because sloppy experiments lead to sloppy results.
- Set target pass/fail goals ahead of time or you’ll be tempted to rationalize what happened after the fact.
- Ask how you can just learn just as much, if not more, with half the time and effort.
- Optimize for learning, not for building product, or you’ll move too slowly.
- For big experiments, do a trial run first because you’ll often discover things to improve.
- Run your experiments with intensity and speed, because time will disappear faster than you think.
- Include opportunities for qualitative research (talking to humans!) as you go.
- Fight your own confirmation biases. In other words, don’t twist results to hear what you want to hear, or dismiss undesirable results too quickly.
- Combine evidence and judgment to make smart decisions (consider running weekly decision meetings) and execute!
For those interested in purchasing Testing with Humans, you can find it on Amazon.