For our last Lean Ignite here in New York, I decided to have a little fun and rant about all the excuses I hear from teams as to why they are *not* doing customer development. Below is the 5-minute video for those who are interested (link). And yes, now I finally understand why people tell me I sound like Jerry Seinfeld…

A lot of customer development language revolves around ensuring that you are solving a real problem, and the right problem.

But what if you aren’t solving a problem?

Lean startup principles still apply to games and entertainment apps. You have the same things to validate: user experience and an understanding of your value, who your customers are (and when certain types adopt), user acquisition, market size, and business model. You can still treat your efforts as experiments.

With an entertainment experience like a game, your initial focus should be validating fun.

You won’t get that from interviews. You validate fun by testing your game.

You *can* learn a lot from interviews. A great game designer studies fun. She plays tons of games, reads the theory, investigates who plays what games, and tries to understand why certain games are winners or losers. But even great, thoughtful designers come up with lousy games.

Where teams go wrong is not realizing that digital games can indeed be tested early.

Paper testing is a critical tool. This is a natural fit for puzzle and board games, but sometimes less-obvious games can be paper tested with a little creativity. It is worth a shot, and usually faster and cheaper than writing software.

Great teams don’t just paper test internally. They get out of their own heads and watch how other people interact with the motivation loops, with the story, and with other players.

Some games are really hard to paper test. Some things do require non-trivial MVPs. World of Warcraft, for example, would be very challenging. But you don’t need all of WoW to know whether you have something good. You wouldn’t need multiple game areas, high visual quality, the full economy, full character powers, a leveling system, or countless other capabilities. All of those features could be added once the root experience was proven to be fun.

And games, just like any other application, should use a mixture of qualitative and quantitative testing to understand user experience and measure progress as changes are implemented. For qualitative, I am referring more to guided observation of actual game play rather than interviews.

The truth about MVPs is that most people think they need a bigger MVP than they actually do. Another classic mistake is that people get caught up thinking about digital product, and they fail to think creatively about analog tests or tests that don’t revolve around *features*.

For your game, think about the smallest unit of fun.

Final thoughts:
With software, there is going to be one core experience or value that captivates your early adopters. You don’t always know what it will be going in, but you want to find it. More features is rarely the answer. It doesn’t matter what you are creating — don’t build a mansion until you have confidence in a great foundation.

There are two established New York tech companies who do a great job of getting close and staying close to their customers: Meetup and The Ladders.

Andres Glusman (VP of Strategy, Meetup) and Jeff Gothelf (Dir of UX, The Ladders) established regular, lightweight usability sessions at their respective companies (that’s them above giving Lean Ignite talks — see bottom for links). Every week they bring in 2 or 3 actual or potential customers to learn about user behavior and mindsets, test out new concepts, and expose current flaws in design or strategy.

I asked Andres and Jeff for their biggest challenge in running a lean usability lab. Both answers boiled down to interpreting, and thus acting upon, learnings. To Andres, the key is balancing different people’s reactions and keeping things in tune with bigger goals. As he put it, you neither want to “freak out and change everything immediately, nor dig in your heals and refuse to change anything. The trick is finding the right rhythm for integrating learnings into the dev process.”

Jeff wrote, “Tempering reactions is the hardest thing. Sometimes it’ll take 12 participants saying the same thing to push an idea through and other times one user can skew an entire session.”

I have felt this first hand, both as a team leader trying to keep a cool head and as a product designer over-reacting to a single user’s pain. It takes discipline and an open mind to handle customer data well.

Here are five tips that might help:

Jeff notes, “Nielsen always said that between 5-8 users are needed to see 80% of the big problems in your experience. Without a dedicated user researcher we don’t have the luxury of doing 5-8 tests per week as it is time consuming and cost-prohibitive. We’ve found that with 3-4 users in every week we can both knock it out in a half day and get enough insight as to what the ‘boulders in the road’ are to give us direction on where to go next.”

Even a couple users every other week is better than what most companies do, which is none at all, or the infrequent binge. The critical thing is getting a continual, lightweight process in place. There are plenty of excuses to put this off, but it all comes down to what you consider important. Just do it.

Examine your biggest risks and test accordingly. Decide what type of customer you want to learn from, and know your core questions or user tasks before they begin. Always make room to understand the customer’s mindset, context and behavior right at the start of a usability / customer development session. As Andres recommends, “with earlier-stage things you want to be more broadly exploratory, while later on you can focus more exclusively on more tactical stuff and straight-up usability.”

You do not need statistical significance if you are seeing a recurring pattern and your gut tells you there is indeed a problem. However, to have a pattern you do need more than one person!

Sometimes a flaw is exposed that you, as a product creator, are dying to fix. We all take pride in our work, and seeing a design flaw can literally, viscerally hurt. But you have to ask yourself where the problem falls in the bigger picture of what the company needs.

Don’t get self-indulgent to mitigate your own pride/pain if there are bigger fish to fry for the company. Especially in the earliest stages of a startup, don’t get lost in endlessly optimizing usability and features if you haven’t solved the bigger problem of finding product-market fit.

User feedback can clarify product debates in a very healthy way. It gives you the “why” behind the metrics you are seeing. It shifts the discussion out of the realm of opinion and ego and into reality. But what do you do if someone really does dig in their heels and becomes a total recalcitrant, refusing to participate in the process but blocking the team?

Andres says, “No matter how good a storyteller you are, you cannot convey user pain as effectively as just watching a user struggle.” If someone cannot observe live, then use video. If a decision is important enough to warrant it, Jeff recommends editing together a video with multiple clips of users laboring to solve a problem area.

Evidence is powerful stuff.

More resources:

• Andres’ great presentation on running a lean usability program (full of great advice)
• Ignite talks from Andres and Jeff
• Steve Krug’s Rocket Surgery Made Easy
• My 12 tips for customer development interviews

I got into a really interesting discussion yesterday where I found myself defending lightweight usability testing. My core point:

Analytics are great for What’s. For example, “A is better than B”, or “this is a problem area”

What analytics do less well is help you understand Why’s. If you don’t understand the root cause, you can end up spinning your wheels or missing big opportunities.

I’m a firm believer in needing both. Analytics keep things grounded in real behavior, but qualitative user interactions provide those needed intuitive leaps.

p.s. for an interesting presentation on user testing, check out Meetup VP Andres Glusman’s deck from last SXSW.

A year ago, I gave a talk at the very first Lean Startup Machine about giving customer development interviews. Tomorrow, I am doing the same with a new batch of LSM warriors and I have revised and updated my list (and accompanying text) as follows:

Focus groups are a group-think, distraction-filled mess. Avoid them and only talk to one person at a time. If desired, you can bring someone with you to take notes — some UX designers like this approach. Personally, I tend to do one-on-one interviews because I think people loosen up and thus open up a bit more.

Have your assumptions and thus learning goals prioritized ahead of time. Decide who you want to talk to (age, gender, location, profession/industry, affluence, etc), and target interviewees accordingly. Prep your basic flow and list of questions. You might veer off the plan to follow your nose, which is great, but go in prepared.

Decide up front if your focus is going to be on learning a user’s behavior and mindset, and/or getting direct feedback or usability insights on a product or mockup. Do not mix the two in the discussion flow or things will get distorted.

Put “behavior and mindset” first in your discussion flow. During this part, don’t let the interviewee go too deep in terms of suggesting features (some people can’t help it), but keep them focused on if they have a problem, how they think about the problem space, and if and how they have tried to solve it in past. Getting people to discuss their actual actions, not just opinions, is very useful.

If you don’t do this, you might find yourself selling or convincing, or even hearing what you want to hear. Remember, the goal in this early stage is learning and validation/invalidation, not a sale.

Unless, of course, you have set a sale or LOI as a goal. You might want to shoot for a commitment from the interviewee as a way to measure true demand. If so, keep it entirely out of the behavior/mindset portion of the discussion.

People are trained not to call your baby ugly. You need to make them feel safe to do this. My approach was to explain that the worst thing that could happen to me was building something people didn’t care about, so the best way they could help me was absolute, brutal honesty.

Do not ask too many yes/no questions. For example, minimize such questions as “do you like Groupon?” Instead ask “what kinds of deals do you look for, if any?” “What motivates you to hunt for deals?” “How do you discover deals?” “Do you get frustrated with the deal sites out there?”

Try to shut up as much as possible, and try to keep your questions short and unbiased (i.e. don’t embed the answer you want to hear into the question). Don’t rush to fill the “space” when the customer pauses, because they might be thinking or have more to say.

Make sure you are learning, not selling! (at least not until that part of the conversation, if relevant)

If you stay *too* quiet, some folks might start getting uncomfortable, thinking that they are boring you or you are judging them. You can keep things rolling with little motions of encouragement, such as nods, “I see”, “interesting”, etc. But do not say things that might steer or influence the interviewee.

Anytime something tweaks your antenna, drill down with follow up questions. Don’t be afraid to ask for clarifications and the “why” behind the “what”. You can even try drilling into multiple layers of “why” (see “Five Whys”), as long as the interviewee doesn’t start getting annoyed.

For important topics, try repeating back what the person said. You can occasionally get one of two interesting results through this. In the first, they correct you because you’ve misinterpreted what they said. In the second, by hearing their own thoughts, they’ll actually realize that their true opinion is slightly different, and they will give you a second, more sophisticated answer.

Another approach is to purposefully misrepresent what they just said when you parrot it back, and then see if they correct you. But use this technique sparingly, if at all.

At the end of every interview, see if you can get leads to another 1 to 3 people to talk to.

The details behind a conversation fade fast, so if you haven’t recorded the session, write up your notes and color commentary as soon as you can. I brain-dump into a shared Google Doc so the rest of the team can see it. (Note: I typically have not recorded sessions for fear of making interviewees more self-conscious or careful, but other entrepreneurs have said to me that, while it takes some rapport-building at the start, pretty soon people forget about a recorder.)

Customer development interviews will not give you statistically significant data, but they will give you insights based on patterns. They can be very tricky to interpret, because what people say is not always what they do.

You need to use your judgement to read between the lines, to read body language, to try to understand context and agendas, and to filter out biases based on the types of people in your pool of interviewees. But it is exactly the ability to use human judgement based on human connections that make interviews so much more useful than surveys.

Ultimately, you are better off moving fast and making decisions from credible patterns than dithering about in analysis paralysis.

ADDENDUM
I want to additionally stress that your goal is not to ask the customer to define the solution. Perhaps this is obvious, but the entrepreneur needs to have the vision to look deep into a problem and come up with the right solution. Don’t ask people what they want, but rather study their behavior for what they do and what they need. To this end, try to get your interviewee talking about specific situations, not abstract feelings and concepts.

When Liz and I first started Aprizi, it was very different in concept from where we eventually ended up. I had become fascinated with Mint and Gist, and people’s willingness to share rich data in exchange for the right value proposition. I started thinking about other places where valuable data existed, but was relatively hard to get to. This led me to email and the realization that access to email receipts could generate an incredible database of product-level shopping information that few folks other than Amazon have.

But while email receipts could be useful to a business, that’s not a value proposition to the consumer. So we thought about possible reasons why someone might let a (trustworthy) company analyze their email. I came up with 7 primary ones: a reward points system (ed. after all my time in virtual worlds, I had virtual currencies on the brain); targeted discounts / coupons; easy access to shipping info; analytics on how you spend money; receipt records and organization; 1-click shopping to buy an item again; and personalized suggestions.

Before we decided to officially start Aprizi, I “got out of the building”, interviewing lots of people and running a survey. One amusing discovery was that men wanted reports on their spending activity, but the women pretty universally said “don’t you dare show that to me!”

In the survey, we asked people to rate interest in those 7 options on a scale of 1-5, from “don’t care” to “need now!”. Here are the results, broken into 3 basic buckets:

You might look at that chart and say, “well clearly you want to focus on deals.” However, I had a discussion with a very savvy retail market researcher who warned me that it could be a red herring. Sometimes, he explained, people feel like they are supposed to answer a certain way, but when you examine their actual behavior, you will see a different pattern.

Indeed, that’s what I found when I did more qualitative research. Now, there are lots of people who obsess over deals, but I was focusing on what you might call “middle-class” customers, since I felt like the low-end of the market was pretty covered on the Internet. I found that deals and discounts were not as strong a driver of people’s actual behavior as one might think.

Note that we didn’t have “sharing what you buy” on that list of value propositions. I didn’t buy it. When Blippy and Swipely emerged and raised a ton of money, I starting asking people about public sharing and got really negative reactions. I think that was confirmed by the lackluster uptake of those companies’ initial products.

We ultimately decided that concrete interest existed in two categories: personalized deals and personalized suggestions.

Liz starting building a prototype that would integrate with gmail and pull in Amazon receipts, and I started paper-testing mockups. We also built a “wizard of oz” prototype which felt like a functioning website but was really me trying out different kinds of recommendations on people. We tested both deals and suggestions, things like sample sales and coupons, commodity products and unique items, different price points, and big brands vs indie ones.

We ran some user tests watching people (at this point, we had already focused pretty strongly on women) use the prototype, or if they were remote, trying to hop on the phone with them just after usage. The highest point of “happiness” was reached when a woman discovered a new store or product that held a designer or brand she had never heard of. The body language was amazing, and we decided to focus on trying to bottle that moment, that experience.

Here’s a slide from one of my early investor decks on our “learnings” from all the customer development work we were doing:

I was also getting a lot of signals that people were increasingly interested in smaller or more socially conscious brands, but that it was really hard to make sense of all the noise in the market. What should be a fun shopping experience online was a serious chore. Our decision to focus on becoming a “Pandora for shopping for independent brands and designers” led to a rethinking of the product and our business model, but that can be the subject of another post.

Ultimately, we generated a decent amount of enthusiasm from thousands of users, but we struggled with the investor community. They respected Liz and my backgrounds, but being mostly men, didn’t personally get what we were trying to do. Aprizi was too complex, the “long tail of design” was too unknown, and we made the fatal mistake of delaying connecting the dots to revenue and instead focused our efforts on iterating the product experience. That might work in Silicon Valley, but is very tough in New York.

I do still think that Aprizi tapped into a big problem and someone, someday, will become very successful by solving it. But I can understand the hesitation of investors as well. We had pivoted our business model three times, and while I thought that we could get it working, I knew that there was risk remaining in our e-commerce strategy. I remain proud of what we accomplished with just 2 full-time people and a mere $35K in capital. Ultimately, having a family, I had a limited runway to make it work. We’ve cut down the servers to a bare minimum, but if interested you can still see the product for a little while longer.

So what was the $9.4M you mentioned in the title, Giff? That’s because I woke up this morning and saw the Techcrunch headline that “Project Slice” had raised $9.4M to — guess what — help you manage your email receipts, the concept we had long left in the dust (including throwing our email integration code in the dustbin).

Now I’ve been in this industry long enough to know that raising a lot of money is NOT an indicator of success, but it inevitably raises the question, did we make a mistake? Did we follow our noses off a cliff instead of into the fruit trees?  Maybe Project Slice will prove our pivot wrong. Offermatic is another West Coast company that has raised a lot of money that could show that we were wrong in our conclusions. Entrepreneur to entrepreneur, I have to wish them the best.

We *are* seeing a trend right now where VCs hand enormous sums of money to entrepreneurs with a “previous win” long before product-market fit is established. Granted, that in itself is nothing new, but the volume is higher than the preceding decade.

Startups are a series of judgement calls, and you make the best ones you can based on the information you have at the time.

I don’t regret the 15 months we spent working on Aprizi, not for a second.

On Friday, I gave a 20-minute talk at a product design conference organized by Ty Ahmad-Taylor (CEO of FanFeedr) and Hard Candy Shell (the talk shared the same title as this post). I discussed mistakes and lessons from Aprizi’s journey. I don’t think it was videotaped, so I am going to take advantage of this flight to California to write down some of the salient points here (and include thumbnails of the slides). Longtime readers will recognize many of these statements.

First, we assumed that the audience had read Steve Blank and Eric Ries. Second, if you are unfamiliar with the term product-market fit, it was coined by Marc Andreessen, and I think of it as basically meaning that you have the right product for the right market at the right time.

I want to stress that this was focused on PRE product-market fit, when you are still iterating on the problem you want to solve and how you can best solve it.  Here are the six points that I covered:
1. Focus on the Value Proposition
2. Qualitative not Quantitative
3. Focus Groups are Evil
4. Own Customer Development AND UX
5. Let Some Things Suck
6. Beware the Siren Song of Investors

I’m going to skip discussing my background in this post — for that, you can click here.

As for the context of Aprizi (currently in open beta), let me ask you a question: have you ever walked down a cool city street and enjoyed discovering a cool boutique with beautiful, unique products? Well, we want to bring that experience and feeling to the Web. Aprizi gives people a fun and (increasingly) personalized way to discover boutiques, independent brands and emerging designers online. We focus on design and lifestyle-centric products. We’re not a marketplace like Etsy or Boticca, but rather a discovery engine.

When Liz Crawford and I started Aprizi last December, we knew that we wanted to make online shopping both smarter and more fun, but it has been quite a journey from that point to the present. Some of our initial hypotheses held up, and some died under the customer development sword. Our journey consisted of hundreds of interviews across an arc of paper testing, manual alpha (i.e. me behind the website as the hamster on the wheel), a crude first beta, and finally now, a baseline beta product which I am really happy to have as our true starting point.

We made some mistakes and learned lessons along the way, so let’s dive in.

1. Focus on the Value Proposition

This sounds like such an obvious statement but teams can get lost thinking about their own needs rather than what the customer needs. Let me highlight with one of my moments of idiocy. At the start of Aprizi, we got obsessed with email receipts. We thought, “there is item-level purchase data locked up in people’s email inboxes — we can parse it and that would open up so many opportunities!” I was doing lots of customer interviews to figure out which were the best opportunities, which was fine and good, but on the coding side, we started building and testing this email infrastructure.

Thankfully, we woke up after a few weeks and realized that building this infrastructure had *nothing* to do with testing what the customer wanted. It had to do with what *we* wanted, and the data we thought we needed. We put the code on the shelf, got our alpha up to test things that the user might care about, and never looked back. Our business has evolved and we have not touched that code since.

I will give you another example, which I saw in effect with a social games company. The team was so concerned about monetization that they built a virtual goods system before they had even properly tested whether their game was fun, or iterated it to the point that it actually *was* fun. Monetization is important, but “fun” had to be their core foundation. They put their needs ahead of the customer.

Many ask “which hypothesis should I test first?” Definitely think through which assumptions are biggest and most risky, but I think it is always wise to begin with a focus on the value proposition to the customer.

2. Qualitative not Quantitative

I’ve said this before — metrics are really important, but the startup echo chamber can over-emphasize their importance in the early days. The closer you get to product-market fit, the more critical metrics become, but they are not all-important while you are still trying to figure out the right product for the right customer.

If you are trying to solve a gnarly problem, you need to look people in the eye, read their body language, hear their tone, and understand their deeper motivations. You need to watch people using your application, or talk to them right after, not stare at sterile numbers. Beware the “local optima” problem (optimizing for a small problem/market). Yes, install metrics and try to think about which metrics you should really care about, but dig into the human side.

On this line of thought, I think surveys are awesome for objective data, but troublesome for subjective data. Example: at the very start of Aprizi, I did a survey of about 60-70 people and got great factual data about their shopping habits, but also got a huge red herring on desired value proposition. By far, the most popular “solution” the survey takers said they wanted had to do with deals/discounts. However, in a classic case of “what people say is often different from what they do,” when I really dug into their true behavior, very few of the people I had targeted actually oriented their shopping behavior around deals.

Aprizi’s true “eureka” moment did not come from metrics. Instead, I was watching a New York City school teacher try out our alpha. We had given her a bunch of different kinds of recommendations to see what got a reaction. She spotted a tote bag from an an independent artist with a small online store. Her face registered curiosity and she clicked through to the store. Then she lit up and I watched her spend 10 minutes gleefully browsing around this woman’s website. At the end, she turned to me with excitement and said, “I *never* would have found this!” At that moment, all these little things I had been hearing and seeing finally sunk in, and I thought to myself, “THAT is what we need to bottle!”

3. Focus Groups are Evil

To properly express my disdain for focus groups, I asked my 5 year old daughter what the most evil thing in the world was. The answer, of course, was Cruella De Vil. And Cruella is actually an apt metaphor, because that’s one of the problems you can get: a dominant personality taking over. OR you get terrible group think. We tried a focus group early in Aprizi’s journey and it really was not effective, at no fault to the participants.

In a customer development interview, you want to start by talking generally about behavior, then talk about a possible solution to a problem, and throughout you want to keep your antenna really sensitive so you can drill down into the “whys” of people’s responses. You simply cannot do that effectively when you are managing a group of people.

4. Own Customer Development AND UX

Don’t let your company and product design become a game of telephone. You need the person designing the UX to be the same person (or people) talking to customers. You do not have to be a graphic design expert — anyone can use Balsamiq, and anyone can look at 5 comparable websites or mobile apps and decide what works and what does not work.

We had a short-lived experiment. I am an adequate UI designer, but not the best by any stretch of the imagination. At the start of Aprizi, I thought to myself, I know some really good UX/UI folks, and they’re willing to help me out, so why not start with the product being *that much better* with the hand of a pro!

Mistake. While this designer was super helpful in giving us a start in the right direction (for which I am highly grateful), the problems quickly became evident. The designer knew tons about usability, but just wasn’t close enough to the user’s problem. Even more problematic was that my thoughts about the product and marketing were evolving far too quickly to do anything but drive this designer absolutely batty. We needed too many iterations, and I knew I had to take back over the design or else threaten a good friendship.

5. Let Some Things Suck

When you are still hunting for PM-Fit, you have to let some things suck. It is hard for people like us, who take pride in our work, but it is necessary. You really could spend a lifetime optimizing the wrong product and go nowhere.

At this super-early stage, focus on learning. Only do what is “good enough” to learn. The definition of “good enough” varies tremendously depending on the product and customer, so you have to use your own judgement here.

In Aprizi’s first beta, we had a web form that I’ve written about before, so going to skip details here. I came to hate this form with a passion. We learned some useful things through the questions asked to the user, and the metrics showed that conversion was not a huge problem, but when I actually watched people get to this page, I saw them stop dead in confusion. Obviously, you never want that to happen; you want to always give people an action and get them to the sexy sauce as fast as possible. So did we fix or remove this page? No we did not. It killed me, but this form wasn’t getting in the way of learning, which was our true goal. Once we had learned what we wanted to learn from beta-1, and got to work on beta-2, this form was happily taken out back and shot.

6. Beware the Siren Song of Investors

I got a lot out of talking to investors early on in our journey. We weren’t trying to raise money, just looking for advice. You can learn about competitors, business model ideas, and the many ways similar things have failed before (VCs never have a shortage in this department). However, for all the well-intentioned, intelligent advice you will get, you need to be careful around the product.

Investors know what is hot right now. That is part of their job. Digging deep into a gnarly customer problem — that is your job.

When we talked to investors, we kept on hearing “you should focus on deals!” Why? Because Gilt, Groupon and Woot were out there killing it. Deals were the rage. However, in startups, as hockey players say, you want to: “skate to where the puck will be, not where it is.”

So my caution is don’t act like a cat chasing a lure perpetually out of reach. Sometimes the new hotness can reveal opportunities — take Yipit, for example — but you need to really think about the problems and customers *you* care about.

The reality is that investors don’t want you to build what they say you should build; they just want you to build something successful.

- – -
So there you go — roughly my six points for the talk.  The folks in the room then asked a whole bunch of great questions, and you should not hesitate to do so either, whether in a direct email to me (giff.constable at gmail) or in the comments below.

There are many strategies for customer development, and here is another one: start a company blog focused on the problem you hope to solve and the people you hope to solve it for.

In this context, your purpose with the blog is not to get vast amounts of traffic, but rather to help you connect with and learn from lots of interesting people. If you do get traffic and SEO, so much the better, but you need to decide whether you want to allocate the resources required to fully chase those things.

With Aprizi we started The Tail. I interview folks in and around our key sectors of e-commerce. I interview shop owners and indie designers and even entrepreneurs focused on this space. We talk about their personal story, successes and challenges. Now, I might be able to get some of their time without the The Tail, but with the blog I give them something back (publicity and SEO). It helps to build the relationship, and relationships make the world go ’round. If you are trying to help someone, they are more apt to want to help you.

You can also use your blog to learn in areas beyond customer development. I wanted to learn more about small business SEM, so put on my blogger-journalist hat and interviewed the super-smart execs at Clickable for a story that could help other small business owners as well.

To make this work, you need the blog to look and sound very professional. Tee up some good first interviews based on existing relationships, so that the next wave of people can look, as they decide whether to do an interview with you, and say, “it might be a small blog, but the quality is high, including the people they have been talking to.”

For those of us who have never been journalists, it is very time consuming. I post once a week, wish I had time for two, and have a backlog of stories to write up at any point in time.

The last thing I will mention is that your blog can also help your relationships with other bloggers in your space. Having now been part of 3 distinct blogging communities, I have seen over and over again that bloggers are welcoming, rather than competitive, to new voices as long as you handle yourself with grace and generosity. It is harder to build relationships with bloggers when you are an outsider. This cannot be machiavellian — it must be genuine, but I assume since you are brave enough to start a business that your passion is genuine and deeply-rooted.

That’s all the typing i can manage on my iPad (writing from the train). Looking forward to talking at the NYC Lean Startup Meetup tonight!

Trevor Owens, the young dynamo in the NY startup scene who organized NYC’s lean startup machine weekend, asked my help in discussing “Idea Validation and Opportunity Assessment” at an all-day NYU event. We have been emailing back and forth about an audience participation exercise and I would love your thoughts on what could make it better.

New entrepreneurs hit three common challenges when it comes to customer development:

1. people worry about their idea being stolen
2. people spend too much time explaining/pitching their idea rather than listening
3. people are uncomfortable approaching strangers, and fear rejection

We are thinking about focusing on the latter two, and creating a practice exercise where participants can role play without the emotional baggage associated with their own idea.  We are thinking about the following 30-minute exercise (still evolving):

LOGISTICS: ahead of time, you randomly pair everyone up twice so that everyone gets to role-play both interviewer and interviewee. To keep things moving, the organizer will start with a roll-call, where he/she says an interviewers name, has them stand up, and then tells the interviewee to walk over to that person. Once people are paired up, they have 10 minutes to engage in an interview (with the interviewer taking notes).  At the 10 minute mark, everyone stops and the organizer runs the second roll call in the same fashion (with the new pairings and reverse roles), and they again have 10 minutes.

CONTENT: You assign a single topic to everyone.  It needs to be a problem that most people have. It doesn’t have to be original, but it should be something not properly solved yet. For college students, it could be getting a date or a job/internship. My thinking is that we actually want to take away the “product idea”, so the interviewer has nothing to pitch and can focus on learning.

We could do a little coaching to everyone before they start, sharing possible questions and explaining the “5 whys” (which one has to be careful not to overuse in an unnatural way). To toss out some examples:

- do you need to do this task?
- how do feel about the task?
- how do you currently handle this task?
- have you tried to find ways to make this task better? Why / why not?
- would you pay to have something or someone improve this task?

And if those start to dry up, there are also interesting general questions to suggest, which help one find commonalities among the people you are talking to, such as:
- what web products do you regularly use?
- what web products do you really love?
- what web products do you really hate?
- how do you typically find out about new web products to try

Extra: have everyone email their custdev interview notes to the organizer. Depending on the topic, it could be really useful data for a real startup out there!
What do you think? How could it be improved?

As promised, here is my 20-min presentation to the Lean Startup Machine event on July 23, 2010. Regarding the event, I was pretty impressed with how much the teams accomplished over the weekend, and their willingness to get out of comfort zones.  This deck is neither as pretty nor as good as David Cancel’s recent talk, but it served the intended purpose of giving the participants a taste of customer development put into practice.  Below is some quick color commentary on the slides:

2. Who am I? Key message: all of the previous startups I’ve been involved with talked to their customers, but that isn’t enough.  “Lean startup” is about ruthless and rigorous testing of assumptions and hypotheses. Think of your startup as a giant stack of assumptions — the more you push testing those conscious or unconscious beliefs into the future, the greater the risk they will come and bite you. I wrote more about previous experiences and lessons here.

3. The Context. Our particular decisions came out of the context of Aprizi‘s customer, product, business model, etc. With that in mind…

4. The first rule of lean startup is…

5. There are no rules. Lean startup is a methodology and framework of ideas, not a playbook or roadmap. Use it to add self-awareness and rigor to your business, but make your choices based on your own context.

Aprizi’s Customer Development Steps (at which point I gave summaries of topics covered in the following older blog posts)
6. Initial Idea Validation. See this post
7. Customer Development. See this post
8. Alpha. See this post
9. Open Beta. See this post

12. Doing a Customer Development Interview. I wrote specifically about this slide here

13. Learning Comes in Waves. Focus on learning, not quotas. What I have found is that with each new step (idea, paper mockup, alpha, beta), we had an initial high burst of learning around some big decisions, and then learning started to become incremental. At that point, our efforts in customer development slowed down because we needed to make decisions, then get to work on the product so that we could get to the next wave of learning.

14. Strip Out Everything But the Core Value Proposition. On Sunday I tweeted “Many features in a young product is often a sign of not understanding the problem or customer well enough.” I was laughing at a ridiculously over-featured early Aprizi wireframe shared with the participants as an example of Balsamiq.  Needless to say, we took a serious scalpel (or hatchet) to the feature list, and of course customer development helped us focus. We simplified and tried to hone in on the point of maximum delight. When you have extremely limited resources, simplification and prioritization is a necessity, but it also just makes for a better product.

I also gave an anecdote from the very start of Aprizi. We built some code to automatically gather shopping preferences and history from a webmail account. We realized that it had nothing to do with testing the actual value proposition to the user, so we quickly tabled it.

15. Pre-PM Fit. It’s All About People. In the very early days, don’t get lost in surveys and A/B tests and optimization or even continuous deployment. That stuff is really important but gets more so as you approach and pass product-market fit. Initially, focus on people. Watch them use the product. Listen to their tone of voice. Watch their facial expressions. Remember, all business is about people. They will use your product, share your product, buy your product, or ignore your product. You need to understand why.

My last message about people was a reminder not to completely ignore business relationships that you will need later on. A bizdev deal, a PR relationship, a blogging strategy, etc, these all take time, and you don’t want to begin from a cold, standing start once you see signs of product-market fit. So focus on your customer and product, but don’t forget to plant the seeds of these future marketing needs.