On Friday, I gave a 20-minute talk at a product design conference organized by Ty Ahmad-Taylor (CEO of FanFeedr) and Hard Candy Shell (the talk shared the same title as this post). I discussed mistakes and lessons from Aprizi’s journey. I don’t think it was videotaped, so I am going to take advantage of this flight to California to write down some of the salient points here (and include thumbnails of the slides). Longtime readers will recognize many of these statements.

First, we assumed that the audience had read Steve Blank and Eric Ries. Second, if you are unfamiliar with the term product-market fit, it was coined by Marc Andreessen, and I think of it as basically meaning that you have the right product for the right market at the right time.

I want to stress that this was focused on PRE product-market fit, when you are still iterating on the problem you want to solve and how you can best solve it.  Here are the six points that I covered:
1. Focus on the Value Proposition
2. Qualitative not Quantitative
3. Focus Groups are Evil
4. Own Customer Development AND UX
5. Let Some Things Suck
6. Beware the Siren Song of Investors

I’m going to skip discussing my background in this post — for that, you can click here.

As for the context of Aprizi (currently in open beta), let me ask you a question: have you ever walked down a cool city street and enjoyed discovering a cool boutique with beautiful, unique products? Well, we want to bring that experience and feeling to the Web. Aprizi gives people a fun and (increasingly) personalized way to discover boutiques, independent brands and emerging designers online. We focus on design and lifestyle-centric products. We’re not a marketplace like Etsy or Boticca, but rather a discovery engine.

When Liz Crawford and I started Aprizi last December, we knew that we wanted to make online shopping both smarter and more fun, but it has been quite a journey from that point to the present. Some of our initial hypotheses held up, and some died under the customer development sword. Our journey consisted of hundreds of interviews across an arc of paper testing, manual alpha (i.e. me behind the website as the hamster on the wheel), a crude first beta, and finally now, a baseline beta product which I am really happy to have as our true starting point.

We made some mistakes and learned lessons along the way, so let’s dive in.

1. Focus on the Value Proposition

This sounds like such an obvious statement but teams can get lost thinking about their own needs rather than what the customer needs. Let me highlight with one of my moments of idiocy. At the start of Aprizi, we got obsessed with email receipts. We thought, “there is item-level purchase data locked up in people’s email inboxes — we can parse it and that would open up so many opportunities!” I was doing lots of customer interviews to figure out which were the best opportunities, which was fine and good, but on the coding side, we started building and testing this email infrastructure.

Thankfully, we woke up after a few weeks and realized that building this infrastructure had *nothing* to do with testing what the customer wanted. It had to do with what *we* wanted, and the data we thought we needed. We put the code on the shelf, got our alpha up to test things that the user might care about, and never looked back. Our business has evolved and we have not touched that code since.

I will give you another example, which I saw in effect with a social games company. The team was so concerned about monetization that they built a virtual goods system before they had even properly tested whether their game was fun, or iterated it to the point that it actually *was* fun. Monetization is important, but “fun” had to be their core foundation. They put their needs ahead of the customer.

Many ask “which hypothesis should I test first?” Definitely think through which assumptions are biggest and most risky, but I think it is always wise to begin with a focus on the value proposition to the customer.

2. Qualitative not Quantitative

I’ve said this before — metrics are really important, but the startup echo chamber can over-emphasize their importance in the early days. The closer you get to product-market fit, the more critical metrics become, but they are not all-important while you are still trying to figure out the right product for the right customer.

If you are trying to solve a gnarly problem, you need to look people in the eye, read their body language, hear their tone, and understand their deeper motivations. You need to watch people using your application, or talk to them right after, not stare at sterile numbers. Beware the “local optima” problem (optimizing for a small problem/market). Yes, install metrics and try to think about which metrics you should really care about, but dig into the human side.

On this line of thought, I think surveys are awesome for objective data, but troublesome for subjective data. Example: at the very start of Aprizi, I did a survey of about 60-70 people and got great factual data about their shopping habits, but also got a huge red herring on desired value proposition. By far, the most popular “solution” the survey takers said they wanted had to do with deals/discounts. However, in a classic case of “what people say is often different from what they do,” when I really dug into their true behavior, very few of the people I had targeted actually oriented their shopping behavior around deals.

Aprizi’s true “eureka” moment did not come from metrics. Instead, I was watching a New York City school teacher try out our alpha. We had given her a bunch of different kinds of recommendations to see what got a reaction. She spotted a tote bag from an an independent artist with a small online store. Her face registered curiosity and she clicked through to the store. Then she lit up and I watched her spend 10 minutes gleefully browsing around this woman’s website. At the end, she turned to me with excitement and said, “I *never* would have found this!” At that moment, all these little things I had been hearing and seeing finally sunk in, and I thought to myself, “THAT is what we need to bottle!”

3. Focus Groups are Evil

To properly express my disdain for focus groups, I asked my 5 year old daughter what the most evil thing in the world was. The answer, of course, was Cruella De Vil. And Cruella is actually an apt metaphor, because that’s one of the problems you can get: a dominant personality taking over. OR you get terrible group think. We tried a focus group early in Aprizi’s journey and it really was not effective, at no fault to the participants.

In a customer development interview, you want to start by talking generally about behavior, then talk about a possible solution to a problem, and throughout you want to keep your antenna really sensitive so you can drill down into the “whys” of people’s responses. You simply cannot do that effectively when you are managing a group of people.

4. Own Customer Development AND UX

Don’t let your company and product design become a game of telephone. You need the person designing the UX to be the same person (or people) talking to customers. You do not have to be a graphic design expert — anyone can use Balsamiq, and anyone can look at 5 comparable websites or mobile apps and decide what works and what does not work.

We had a short-lived experiment. I am an adequate UI designer, but not the best by any stretch of the imagination. At the start of Aprizi, I thought to myself, I know some really good UX/UI folks, and they’re willing to help me out, so why not start with the product being *that much better* with the hand of a pro!

Mistake. While this designer was super helpful in giving us a start in the right direction (for which I am highly grateful), the problems quickly became evident. The designer knew tons about usability, but just wasn’t close enough to the user’s problem. Even more problematic was that my thoughts about the product and marketing were evolving far too quickly to do anything but drive this designer absolutely batty. We needed too many iterations, and I knew I had to take back over the design or else threaten a good friendship.

5. Let Some Things Suck

When you are still hunting for PM-Fit, you have to let some things suck. It is hard for people like us, who take pride in our work, but it is necessary. You really could spend a lifetime optimizing the wrong product and go nowhere.

At this super-early stage, focus on learning. Only do what is “good enough” to learn. The definition of “good enough” varies tremendously depending on the product and customer, so you have to use your own judgement here.

In Aprizi’s first beta, we had a web form that I’ve written about before, so going to skip details here. I came to hate this form with a passion. We learned some useful things through the questions asked to the user, and the metrics showed that conversion was not a huge problem, but when I actually watched people get to this page, I saw them stop dead in confusion. Obviously, you never want that to happen; you want to always give people an action and get them to the sexy sauce as fast as possible. So did we fix or remove this page? No we did not. It killed me, but this form wasn’t getting in the way of learning, which was our true goal. Once we had learned what we wanted to learn from beta-1, and got to work on beta-2, this form was happily taken out back and shot.

6. Beware the Siren Song of Investors

I got a lot out of talking to investors early on in our journey. We weren’t trying to raise money, just looking for advice. You can learn about competitors, business model ideas, and the many ways similar things have failed before (VCs never have a shortage in this department). However, for all the well-intentioned, intelligent advice you will get, you need to be careful around the product.

Investors know what is hot right now. That is part of their job. Digging deep into a gnarly customer problem — that is your job.

When we talked to investors, we kept on hearing “you should focus on deals!” Why? Because Gilt, Groupon and Woot were out there killing it. Deals were the rage. However, in startups, as hockey players say, you want to: “skate to where the puck will be, not where it is.”

So my caution is don’t act like a cat chasing a lure perpetually out of reach. Sometimes the new hotness can reveal opportunities — take Yipit, for example — but you need to really think about the problems and customers *you* care about.

The reality is that investors don’t want you to build what they say you should build; they just want you to build something successful.

- – -
So there you go — roughly my six points for the talk.  The folks in the room then asked a whole bunch of great questions, and you should not hesitate to do so either, whether in a direct email to me (giff.constable at gmail) or in the comments below.

I just piggybacked on a twitter conversation between Sean Ellis and April Dunford talking about product management versus product marketing (see April’s post). Sean tweeted this comment which I just wanted to highlight:

“Functions” is part of the problem in early stage. IMO goals better: PM fit, then conv eff, then growth… [Ed note: PM fit = product-market fit; conv eff = conversion efficiency]

I completely agree with this.  At the beginning, you need a small, tight-knit, complementary team that ignores function and focuses entirely on goals, splitting up tasks to effectively meet those goals.  With Aprizi, I do any product marketing tasks, and Liz and I split various product management functions.  Function definitions are not relevant, and we don’t feel pressure to do things irrelevant to the near-term goal of product-market fit, save for some effort sowing seeds via relationship building.

“Function temptation” is more of a problem for heavily- funded companies with large starting teams and immediate investor pressure for growth.

At founding, make sure you all agree on priorities.  I was talking to a very talented business development person the other day, and he was stuck on the importance of distribution.  His logic, as I understood it, was that customer acquisition is the biggest struggle for consumer Web startups, so clearly distribution is the most important thing and must happen right away.

I disagree.  A big distribution deal before product-market-fit is a recipe for disaster*, and I’m not talking about the economics.  You might get pressure from the big partner to make what *they* want, not what the customer wants.  Your flexibility to pivot will be severely restricted by expectations, promises given, and a legal document.  Instead of iterating your product in obscurity, and thus relative brand safety, you risk giving hundreds of thousands of people a bad experience.  That bad user experience will threaten your relationship with the distribution partner and possibly neuter future opportunities.

This BD person isn’t an idiot, but rather they are a talent, attitude and skillset that should come on board only *after* product-market fit is found.

My belief in customer development > product market fit > conversion optimization > growth is borne from many scars, mistakes, and a burning desire to do this thing we call “startup” better.

[Update: just to clarify, I'm not saying that you should completely ignore all BD work, since channels are indeed important and they take a long time to develop. I write more about relationship building in the comments below where Sean and April weigh in, and where I think the discussion is better than the actual post.]

Here are a few thoughts on product-market fit that came up when I was chatting this evening with Brant Cooper of Market by Numbers.

Revenue & Testing
To me, “lean startup” and “product-market fit” boil down to rigorously and continuously testing your assumptions as early as possible, and holding off heavy investments in scalability and growth until you feel confident in the drivers of your business.

Your revenue model is one of those assumptions, but it doesn’t have to be the very first thing empirically proven.  I am thinking specifically of consumer-focused services with variations of freemium (for example, Evernote or Zynga) or business models which separate the end-user and the revenue source / payer (for example, Daily Candy or Google).

In these cases you can build early confidence in your plan with customer development conversations (in the latter case above customer = both users and payers), as well as close observation of customer (or payer) behavior with similar products.

Take the example of a game which makes money from virtual goods.  You have a simple foundation — the game must be fun. Without that, you have nothing.  So you can first test fun, then virtual goods compulsion loops and most likely viral behavior, and then finally whether people will indeed pay for those virtual goods.  The key is that you are testing assumptions all the while, rather than waiting to do it all together after a longer product dev cycle. You are carefully choosing the order of what you test (for example, a more utilitarian product might want to test monetization earlier).  You are keeping an open mind and an eye out for the necessity to change/pivot.

The RIGHT Product-Market Fit
I like Sean Ellis’ target that 40% of your users should say that they would be “very disappointed” if they could not use your product.  It feels like a good stake in the ground.

I think that you must consider customer demographics (UPDATE: I’m using the word demographic loosely here — it’s about understanding customer commonalities and segmentation) when you do this kind of testing. Who is making up this data set and how does that fit into your assumptions?

If you were shooting for teens or 30/40-something women and you end up with the TechCrunch set all jazzed up, you need to ask yourself: “Should I give up on previous assumptions and instead aim at the TechCrunch set? Do I believe that I can effectively monetize that type of customer? Will I get stuck here, or will I make a leap to other types of users?” And if those answers are not coming back positive, “Am I willing to risk losing that momentum by changing my product to make it more compelling to mainstream users today?“  As with all this stuff, there is no universal right answer. Every startup has to navigate their own way.

Chris Dixon had a great post on this called “Techies and normals“.  I don’t believe that “techie” defines “early adopter” to the exclusion of all other types of people.  Technology is too pervasive in our culture now to be that simplistic.  You will have an adopter curve among “normal” (i.e. more mainstream) users too.

Steve Blank writes about not getting ahead of yourself by designing a product for the fat part of the adoption curve, and his argument makes sense.  You do need to think about your early adopters, but within a target demographic.  As always, this is about testing assumptions and being ready to pivot, either on design or on target market, if your assumptions were wrong.

(image from tony newall on flickr)