I find myself thinking about this morning about sources of learning, and how and where one challenges oneself to learn. Are you satisfied with your expertise in your domain, or are you constantly pushing to improve? As I wrote the other week, are you looking outside of your domain?
I watched an experienced market researcher do a customer interview the other day and was surprised to see them ask speculative questions such as, “what do you think of the potential product?” and “if you bought this product, what would you expect you would be getting?” I’ve long since consigned speculative questions to the dustbin, as I have with focus groups — both for the same reason: believability.
This person, who was very smart and experienced, said afterwards that in the hands of a skilled researcher, you can get a lot of valuable information. I’m sure that’s true, and this was indeed a master craftsperson.
So I asked myself, “why is this still bugging me?” And I realized that the reason why I no longer ask speculative questions is two-fold: 1) I find it requires too much work to parse out the believable from the non-believable in what you learn; and 2) I have a better alternative in the form of experiments.
I use interviews (and I think other ethnographic methods apply as well) to dig into people’s lived experiences and to try to uncover their goals, motivations and emotions. I use experiments if I want to see what people will do.
If you don’t have experiments in your toolkit, you make the most of what you do have. Some information is better than none, as long as you use your judgement. But the more you can question and improve the believability of what you are learning, which ties very closely to how you are learning (methods, execution, bias in data, etc), the better. This gets easier if you collect more methods, more frameworks, etc. Then you get to a new context and you think, “ah, for this situation, I should use X!”
However, back to my original point: the more we challenge our thinking with other perspectives the better. I don’t look down on this market researcher — I respect them and have much to learn from them. They might have an insight into how to make those speculative questions more “believable” that I’m missing. Perhaps the fact that I have experiments in my toolkit is both an opportunity but yet also limits my growth in the craft of interviews? Is it a double-edged sword?
This morning I finished Gib Biddle’s essay on NPS. I’ve historically stayed away from NPS. It felt too squishy, and I’ve preferred things like the Sean Ellis “very disappointed” test. Actually, I tend to stay away from surveys in general. But I was glad to read Gib’s common-sense essay. It was nice to challenge my own biases.
It’s why I’m trying to make time to read Will Larson’s An Elegant Puzzle: Systems of Engineering Management. It’s why I snap up anything Josh Seiden publishes, because he thinks about many of the same problems I do, but comes at it from a design perspective.
I’m as prone to moments of pride in my expertise as anyone. I don’t share the above to pat myself on the back but just to share concrete examples, since that’s always nicer than vague hand-waving. I have to work to spot my own bias, nix my own hubris, and elevate intellectual humility. When I’m successful, I grow faster.
So ask yourself, how are you reaching outside of your expertise and your craft to learn? How are you adding new perspectives, ideas, and methods to your toolbox? How are you preserving that beginner’s mind while being justifiably proud of your abilities?
Top image by Michal Vrba on Unsplash