If you are trying to get your team to think more creatively in terms of experiments, this exercise might be useful.
Why? When it comes to creating new products (or even features), experiments help you make more informed decisions. Ideally you make your experiment as small as possible to get believable insights, but they come in all shapes and sizes.
My favorite way to get lots of experiment ideas out on the table borrows from the UX world, using what is typically called a “design studio” or a “charrette.”
A design studio is a rapid sketching exercise that helps people shed inhibitions and not overthink. It mixes both individual thought (diverge) and well as group collaboration (converge), which is important to avoid groupthink.
Step 1: list risky assumptions
Before you can design an experiment, you need to know what risky assumption(s) to test. Gather the team together and list out all the assumptions you currently have that, if proven wrong, would cause your initiative to fail. Then cherry-pick the scary ones, i.e. the risks that could have great impact on your business and where you have very imperfect information.
Step 2: prep
give everyone a “six up,” i.e. take a sheet of paper (I like using 11×17″ sheets) and divide it into six sections either by drawing lines or folding the paper in half one way, and then into thirds the other way.
Step 3: narrow down the problem space
Either assign the riskiest assumption, or let people pick from your top 3 or 4
Step 4: sketch
Everyone has 5 minutes to come up with as many experiment ideas as they can that could shed light on that big assumption. Using words and images, they should put a different experiment idea in each box on their page. *Don’t* let anyone get bogged down in one idea. *Do* give them a couple more minutes if they need it.
It can be helpful for people who have never done this before to explain that you are looking for any kind of activity that could help you understand if you are all right, or wrong, about the assumption.
I also tell people to think of an experiment that might take a month to run. Then think of one that might take a week. Then think of one that might take a day.
In my experience, people usually start with excessively large experiments and need practice getting them smaller.
Step 5: share
Everyone will get a few minutes to explain their ideas to the room. During this process, give everyone something to write on, and ask people to jot down the experiments they find most compelling as the presenters talk.
People can ask clarifying questions, but during this sharing process, ask everyone to refrain from discussing what they think is a good versus bad idea. You will likely need to actively moderate for this, but it is important.
After someone is done presenting, tape their six-up paper to the wall.
Step 6: dot-vote
Give everyone three votes for the ideas they think are best. They can use the votes however they choose: across three ideas or all on one. Voting can be done simply by stepping up and putting a dot (ideally in another color) on the box of the six up listing the experiment idea.
While this happening, mentally link up the ideas that are similar, so that you can combine their dots.
Step 7: choose the top ideas
Combining dots that are spread across similar ideas, pick out the top ideas. Choose a number of ideas that is half the number of people doing the exercise, because the next step is to pair people up.
Step 8: refine and define the experiment
Split the group into pairs. The job of each pair is to take the nugget of an experiment idea and refine and define it. Each team needs to list out:
- the assumption/hypothesis being tested (“we believe ________”)
- what will be measured to prove or disprove the hypothesis
- what the numerical success and failure criteria are for that metric(s)*
- who will participate in the experiment
- how you will recruit those people
- how the experiment will be run (i.e. the activities)
- how long the experiment will run for
* bigger experiments, such as a concierge or wizard of oz test, can explore multiple metrics.
Step 9: share and discuss
Each pair should then present the details of their experiment idea. This time, the audience is allowed to provide suggestions and critique.
Step 10: action
Hopefully this process has created a few ideas that you actually want to try. The point isn’t to create sketches on paper. The point is to generate actionable data, so get to it! As you start implementing your chosen idea(s), they will likely need further iteration, but this design studio is a great way to get things going.