I just wound down a project where I spent 6 months working hand-in-hand with Pivotal Labs, the respected agile/XP dev shop. It was a fabulous experience that had a strong impact on how I think about product development (more on that later). It also has changed how I interview product and UX people.
Pivotal helped our mutual client interview developer candidates, and the most critical component was the pair interview. The developer hiring process started with a resume (although in this day and age, a referral and an active github account might be all you need). Then there was a short technical interview. If that was all rosy, then the candidate spent several hours pair-programming with one of our team on the real codebase.
Pairing on something real lets you not only see how the person solves practical problems (even if they are relatively new to a codebase and a particular language), and how exposed they are to practices like test-driven development. Pairing also gives you a better taste of the candidate’s personality.
I started doing UX/UI interviews this way. The first half would be a traditional conversation where I probed about previous work, skills / interests, and whether they had actually “gotten out of the building” to talk to customers. For the second half, I would pick a tricky problem that hadn’t been solved yet by our team, and the candidate and I would get in front of a whiteboard to talk solutions.
My goal was never to see if the person had the “right answer”. That is too much to expect when someone is brand new to a complex problem. Instead, I wanted to see if they had (in no particular order):
1. the action-oriented confidence to grab the bull by the horns, embrace the exercise, and deal with the risk of “getting it wrong”
2. the humility to ask questions
3. the intelligence to ask the right questions
4. the adaptability to take my answers, merge them into their thought process, and come up with solutions
5. the communication skills to interact effectively even when floundering (a certain amount of which is to be expected)
6. a personality that would fit our team
7. the visual and UX savvy to suggest truly interesting solutions
I also did something similar as we looked for a full-time head of product / senior product manager to replace me at this early-stage startup. Again, the first part would be a conversation, but then I would switch to two exercises. The first was to prioritize a bunch of index cards with high-level feature/infrastructure stories on them. The second was to try to detail out a few key low-level stories for a new problem/feature (and in the process trying to simplify, balance scope and effectiveness, and identify risks and dependencies).
As with the UX role, I was probing probes for the right mix of confidence, humility, intelligence, adaptability, communication skills and personality. These product exercises also helped me gauge whether someone could think strategically as well as dive into the weeds and get really practical — an essential skill range for an early-stage startup.
Final Note:
It was surprising how many candidates did everything in their power to avoid engaging with these exercises (it felt like 4 out of 5 tried to duck, and these folks didn’t warm up to it either even with assurances that there were no wrong answers). But early-stage startups are about action in the face of imperfect information, and I think these interview techniques offer a greater window into someone than conversations or brain teasers.