My Answers to First Round Network Product Questions

First Round Capital has a wonderful online network, basically a white-labelled quora, for people at their portfolio companies. I answer questions now and then. Here are some of the Q’s I have answered over the last year or two:

How do you best manage expectations/predictability when interfacing (roadmap, status updates, etc) with the company outside of product development?

With our (Axial’s) B2B online marketplace, we don’t have to make commitments to customers long beforehand, so our roadmap can stay pretty flexible. And we take advantage of that because we are constantly tweaking it based on what we’re learning. The answer to your question for us has been three-fold: a lot of externalization, a lot of interaction, and finally, building trust in both our pace of delivery and our decisions.

For externalization and interaction, we try to run brainstorming sessions with the different teams (customer success, sales, etc) every few months so that teams feel heard. We write an internal blog post almost every week that talks about what is being worked on, why, and any roadmap changes, and why. We also publish important research findings, metrics or experiments. Not everyone reads everything, but by not being a black box, we build trust. We also constantly try to draw lines back to the key “north star” goals/OKRs of the business. And we’re quick to respond to bug reports and questions. Again, no black box.

We do set expectations around timing, and we view those as goals but not hard promises. The team has learned that we’re pretty good within the next 2-3 months, but anything can change beyond that time period. It helps that we have a culture that buys into shipping things that have *impact* rather than sheer code output — even when that impact is sometimes more qualitative than quantitative.

Finally, we’ve been able to create room for flexibility because 1. our colleagues know that we agonize over what to work on with our limited throughput, and 2. because we ship every week, multiple times a week, and so it feels like we’re making continual progress.

There’s no freakout if something slips, not because everyone really gets the true complexity of software (they don’t), but because there is trust and our teammates now give the prod-eng team the benefit of the doubt.

As a Product Manager or Leader, What tips can you share to build strong relationships with engineers on your team?

Here are a few guidelines:

  • Treat engineers like creative partners, not code monkeys, which means involving engineering early
  • Communicate the why, not just the what, and do so with humility and an openness for ideas
  • Be an effective $hit (or shiny object) umbrella, insulating them from the billions of internal product opinions and senior exec whims
  • Respect “flow” and try to minimize context switching
  • Write clear, concise user stories that don’t presume to tell engineers *how* to implement something
  • You don’t need to be an expert programmer to earn their respect, but you do need to be fluent in how the sausage is made. What is the release process and why? how do you handle QA and why? how is the team investing in devops and what is that enabling? etc.
  • Invest time in spending time with the team outside of work

And the most important one: never ever ever bullshit an engineer

What career paths are there in the product world for non-technical product managers?

There are few limits these days to a non-technical product manager. As the PM role continues to shift to an “outcome manager” role rather than a “make sure engineering is productive” role, a comp-sci background is no longer as relevant.

There are exceptions. It’s very difficult to be a non-technical PM if your end customer is highly technical (an engineering product, for example). In addition, some companies still have (arguably outdated) biases here, but I suspect that is shifting to the minority.

It’s also worth investing in becoming more technical, because it will help you communicate with and earn the respect of your technical teammates more effectively. It will help you ask good questions. But ultimately, they’ll respect you because you’re a good PM. No amount of CS knowledge will salvage respect for a bad PM.

The first place to work on technical fluency is understanding the production process — all the details in how the team gets something from an idea to something live. Do you do automated testing? pairing? code reviews? a certain approach to version control? There is context and why/why nots behind every one of those decisions.

Then it’s worth hacking up your own mini application as a side project where you wear all hats – PM, designer, and coder.

The goal in investing in your own education here is not to become a full-fledged software developer (or designer), both of which take a huge amount of investment to gain true proficiency, but rather to build your own awareness. As a PM, your goal should not be making technical decisions, but becoming informed enough that you can ask the right questions.

What’s your favorite tool for product roadmaps if you already have a ticket tracking system?

Our toolset is constantly evolving but currently:

  • Trello to manage the roadmap from a high-level business perspective, including the goals and experiments we want to run. In my experience it is necessary to have a place for people to go look, but most don’t actually do so.
  • Jotto, an internal blog, to share weekly updates on quarterly OKRs, what is in the works and soon to ship, evolutions in strategy, interesting metrics, interesting external research notes, results and iterations of recent product initiatives, etc across the company
  • Google Slides to actually edit/manage the OKR sheets (we have used the OKR quadrant Wodtke talks about in her book Radical Focus) – but this is more internal to the team
  • Pivotal Tracker to manage our in-the-weeds work across the different teams (again internal to product/engineering team)
  • A wiki for technical/engineering documentation (internal to prod-eng team)
  • Some of the most important work is the lo-fi / lo-tech stuff, doing collaborative sketching (design studios), bi-weekly qualitative feedback/research reviews, roadmap planning sessions etc with different groups across the company. For major roadmap updates, there is a larger presentation across the company.
  • We use Google Sheets if we are doing impact/effort/risk weightings across different ideas, and Excel for larger, more complex modeling

One exercise I recently ran was as simple as writing down all the lurking product ideas on 50+ white index cards and having each team do their own force-rank card sort. It’s much easier for people when it’s tactile, but that does require the group being in the same room. All the teams could see the results of the other teams (as simple as sharing the photograph of the final card sort on Jotto). To be clear – we didn’t make the final roadmap a democratic process, but the product team appreciated hearing the different perspectives and everyone appreciated being heard. If you do something like this, it’s often worth doing a separate one with the executive team.

Regardless of the tools, it just takes a lot of engagement and repetition.

Who runs user testing at your startup? What have you seen work or not?

My belief is that user testing and user research is a constant thing across the lifecycle of all product work, which means each team is responsible for doing it. One also needs to be practical — the shape and cadence of research depends on the nature and risks of the idea or problem/goal/feature/strategy you are investigating.

So I’m not a fan of centralized usability or research (or design service centers) at startups. Market feedback needs to be visceral/personal, very “first person” to the team working on the problem/outcome. It’s important to also figure out how to get engineers exposed to the process and feedback.

One exception to this is recruiting, particularly for consumer products. Having a centralized responsibility for ensuring 1. a constant flow of customers/users and 2. smooth logistics both for the users as well as which team gets to talk to whom, can bring extra efficiency to the process. Meetup, who took having a constant cadence of user feedback to the next level, took this approach to good effect.

I do think this starts to change as you scale to a very large product team. Building a research competency can ensure better training of your product team, a better interviewee experience, and better externalization of “learnings” across the org. But even in this situation, the product teams themselves need to be fierce about maintaining their own pipeline of first-person knowledge.

What quantitative metrics can help assess product-market fit?

Sean Ellis asks customers how disappointed they would be to no longer have your product. Working across a lot of different startups, he came up with the heuristic that you want to see more than 40% be very disappointed.

David Skok also recently did a long post on this topic.

The hard truth is that an early semblance of PM-fit can often be a mirage simply because of the dynamics of the technology adoption curve and the fact that customer types are changing as you try to scale. It’s why the book Crossing the Chasm is still relevant so many years later. While I really like Sean’s framework, and actively put it into practice, I think every startup has to zoom into its own context and its own key indicators of scalable, repeatable growth and business model success. Enterprise (B2B), consumer (B2C) and marketplaces all have different characteristics. It can be brutally hard in the early stage to uncover your consistent and authentic predictive metrics, but worth the effort.

One of the biggest mistakes when it comes to examining PM-fit is focusing excessively on growth numbers rather than retention/churn numbers. Startups are often good at goosing/driving growth, but the real question is whether the foundation is solid and scalable.

What do you look for in your first VP of Product or Head of Product hire?

If you are the CEO, the first thing is to look within:

  • Are you ready to let go of product?
  • Are you looking for / ready for a thought partner, or do you want someone to primarily make the trains run on time?
  • Do you already have strong beliefs on how one prioritizes and ships product?
  • Do you already have strong beliefs about the importance of intuition vs market data/learning?

Then you need to look at the org:

  • How much alignment versus controversy is there in terms of mission and priority?
  • Are you still hunting for product-market fit, or are you now optimizing and extending?
  • What is the scalability of the engineering or design leaders? (i.e. do you need a product leader who also needs to take on engineering and/or design for at least the near term)
  • What are the values of the engineering organization?

Your answers to the above are necessary inputs into the personality, values, creativity, experience, and seniority of person you recruit.

The above stuff is where things can get messy if you get the fit wrong.

It’s usually a given, IMO, that you also want someone who can:

  • inspire and recruit great people
  • closely align product management, design and engineering
  • highly comfortable with data and business models
  • become an expert in your domain (either through current knowledge or a proven track record in being a quick-yet-deep study)
  • communicate and build bridges effectively up, down, across and out
  • ship
  • get results

(other thoughts on good vs bad prod mgt)

What are best practices for including and leveraging the design team in your product development process?

I boil it down to two key essentials, and I think a lot of modern software teams are moving to this model:

1. Avoid running the product design team as a service center.  Instead I believe in embedding dedicated designers on small cross-functional teams. As you scale up, you might find that it makes sense to centralize an additional research capability, but that still doesn’t preclude dedicated, embedded design on teams.

2. Hire designers who think about design with a capital-D, i.e. they care as much about understanding and solving the customer’s goals as they do visuals and usability.

As your teams are running to hit whatever goal that has been set, ideally they are doing a mix of research and experimentation as well as building, shipping and measuring. Designers bring a different perspective to the problem/solution process from PMs and engineers. It’s powerful to have that in the mix and leads to better decisions. So essentially, get design involved from the start right through final validation that a feature actually worked.

Note: Even though, in this model, designers and PMs are distributed across teams, they still want to report to a senior member of their craft that can help drive excellence across all the teams.

How does your company create an inclusive culture for those that work remote?

Tools and discipline. A lot of this can be handled at the team level, but it helps to have executive buy-in and support. There’s a nice rule I once heard which was that if one person is remote, you should act like everyone is remote. Hard to live by, but worthwhile to aspire to meet.  This requires real diligence. Either do remote and take it seriously, or don’t do it at all. For example, my engineering team at my current company had only one remote engineer. That wasn’t healthy. I needed to decide whether have more than one, or none. I went with “more than one”.

Some things to stay disciplined on:

  • If you are in a big room, make sure everyone is speaking loud enough to be heard.
  • If you are in a big meeting, make sure you explicitly call on the remote folks to speak (it’s very hard to break in while remote)
  • Get teams to externalize more using an internal blog or something similar – what are their goals and challenges? how are they doing? where are they going?
  • Have your teams try to minimize in-person water-cooler conversations about the work and make sure to invite the remote teammate to the chat
  • Try to bring remote people together (i.e. fly them in) with the team once a quarter
  • Invest the time and money to make the audio/video easy and effective
  • Use video more, audio-only less (i.e. conference calls)
  • Choose your tools wisely. For example, when we do in-person agile retros, we use sticky notes and whiteboards. When we have one or more person remote, we switch to a shared google spreadsheet.

Tools I like:

  • Zoom for video (although Google Meet seems to be getting better)
  • Chat 170 speaker/mics if you don’t have fancy A/V systems
  • Slack for chat (and now Screenhero)
  • Google docs/spreadsheets
  • Jotto for private, internal blog (disclosure: a pet project of mine)

Don’t underestimate the importance of in-person bonding. It keeps teams functional and helps whole startups through the ups and downs. I totally agree with the previous comments above having a clear mission / north start, but even with that, you need in-person reinforcement. I once was in an 85-person that was almost entirely remote, and it was necessary for us to bring the entire company together 3 times a year to keep everyone aligned with the mission and knitted together.

Given my startup is too small to need managers, what can I do to gain management experience now and position myself to be an obvious choice when we do grow to need managers?

Leadership is less about telling people what to do and more about helping a team be their best. You don’t need reporting lines to be able to practice and build a leadership state of mind. Some questions to ask yourself and work on:

  • Are you raising the optimism and morale of the team?
  • Are you actively helping to ensure shared focus and shared understanding across the team?
  • Are you helping the team set smart goals?
  • Are you thinking about ways to not only improve your own skills but the skills of people around you? most importantly, are you taking action on those ideas?
  • Are you mentoring, collaborating, challenging, and inspiring the team to do better work? are you doing this in a way that does not come across as arrogant – meaning that you are a good listener and open to others’ ideas?
  • When you interview job candidates, are you setting a really high bar? are you thinking about finding someone with new perspectives? are you thinking about how the person will fit in as a puzzle piece to lift up vs weight down the team?
  • Are you thinking about the impact and connectivity of the team on and to the rest of the organization?

You should begin to lead through influence before you actually get direct reports.

How do you balance working on long-term larger projects with short-term quick fixes at your startup?

We currently operate outcome-oriented teams. We keep the teams focused on their goal (KPI) but do weave in critical/major bug fixes. If a small feature request comes in, we first ask if it would contribute in a meaningful way to the goal. If not, it likely goes on the backburner, but we will ask the question of whether there would be enough outsized lift for the amount of work involved that it would justify breaking the flow of the team. Making an exception is a judgement call, but one should always be open to exceptions as long as they don’t become the rule.

It might come out of the icebox in one of two ways:

1. a team gets assigned a new goal where the feature is aligned and worthwhile;

2. we do a “small but good improvements” sprint between major goal initiatives, just as we might do a focused bug-fix or tech-debt sprint. In any of those circumstances, you need to do a fierce prioritization effort, but it can be useful to steal a march here and there that let you stay focused the rest of the time.