In 2010, I became minorly Internet-known as an early adopter of lean startup. I blogged as I went. I made plenty of mistakes. One was not defending adequately against my own biases.
I have a soft spot for entrepreneurs of any kind. That’s one of my big honking biases. Liz Crawford and I were working on problems around product discovery. With an early MVP, we saw a ton of excitement from our alpha testers when they discovered a brand or designer they had never heard of before.
“Follow the heat!” I thought.
Since one of my passions is about supporting the entrepreneur, the notion of helping new designers find a bigger market was right up my alley.
I just needed to be careful. Entrepreneurs need to work on something they are deeply passionate about, but they also need to question their ideas with clarity.
Our first MVP was entirely manual behind the scenes, but rather than continuing to question our assumptions, we jumped to product. We recruited cool curators from around the country who loved to spot new talent. We built a bookmarklet to automate how curators submitted their finds and get it into our discovery database. We built Pandora-like smarts into the engine so that we could personalize items based on people’s favorite curators and likes/dislikes.
We worried about how big the indie market might be, but we didn’t try to test this. We also worried about the conversion rates to purchase with indie products, but decided to put aside business model testing and go for user growth because we thought that would attract investors more. Honestly, I think the increasing pressure to raise money made me make some worse decisions.
Even though we had high engagement and month-on-month doubling, raising money was not in our cards. There was a mix of reasons why, but while we saw decent growth, it was not as explosive as it needed to be. And when Liz and I started consulting for Birchbox, we saw the difference between their explosive growth and our pretty good growth.
In retrospect, I know that we jumped to “product” and “grow it” mode too soon. My bias towards this idea allowed me to conflate customer excitement into a stronger signal than it actually was.
When I talk about customer development and “Talking to Humans,” I often get asked how we can control our biases.
During qualitative research, even if you keep the customer from speculating about a hypothetical future, our brains are wired to jump to conclusions that match what we hope is true.
Some people try to solve this idea by explicitly trying to kill their ideas. If an idea is truly a “weed” (as Fred Wilson calls unkillable ideas), then it will refuse to die.
But that’s hard to do. Entrepreneur passion and a dose of irrational determination to do something completely new is a necessary ingredient.
It just needs to be controlled.
The other approach is the test ideas with multiple vectors into the problem.
You can’t just do qualitative interviews.
You can’t just do a landing page test.
You can’t just do a manual concierge.
You can’t just do an MVP.
You need to do multiple experiments. Some should be qualitative. Some should be behavioral. By checking and cross-checking, you will get closer to the truth.
The advanced version is to examine each test for possible biases, and design the next test to fight against that bias.
This all sounds highly unromatic, but I’m long past the romantic mythology of startups. However, I’m not done with passion and obsession, and I will probably never stop being an entrepreneur no matter how old I get. But it’s a hard game, and you need to steel your mind to be just as hard.