I was asked to give a talk to a group of CTOs recently. Going into the interview, they asked four questions:
- What makes a good, modern PM?
- What are the ingredients for a strong product organization (prod-eng-design)?
- How can engineering help?
- How should a good, modern PM organization think about metrics in 2021?
I tried to push past some of the more obvious answers and below are the notes I wrote for myself going into the session.
What makes a good, modern PM?
There are lots of “good vs bad PM” lists out there (I even created one 5 years ago). Instead, I want to share my opinion through three statements.
- A few years ago, I was speaking at a conference and an earlier speaker got up and said, “The job of a PM is to make decisions.” My brain went, “Wait, what?” And when I got up, one of the first things I said was, “The job of a PM is to ensure that good decisions get made, but not make all the decisions.” That off-the-cuff statement has been a useful framing for me ever since. The PMs that can understand and deliver on this go far. The ones that can’t, I won’t keep around.
The implications of this statement are wide. For sure it’s a required blend of leadership ability and humility. I also think of the job as one of synthesis that leads to good decisions. You’re working from the top down — how do you get from mission/vision to strategy to company priorities and goals to team priorities and goals to the intimate details of the work. You’re also working from the bottom up, with customer research, quantitative data, experiments, and the creative brilliance of your team. The PM goes wide to bridge these worlds, allowing other members of the team to go deep. It takes the ability to change altitudes quickly and well.
This is why so few people can do the job well. Some people say that means the job is poorly designed — an impossible job. I disagree, but do think we need to change how we spot, hire and train PMs. The problem is so much of the job is so-called soft skills, and those are hard to teach. I’m a huge believer in internal transfers into product. - The second statement comes from Francis Frei & Anne Morriss in their book Unleashed: “Leadership is about making others better, first as a result of our presence, and then in such a way that it lasts into our absence.” Good PMs understand and embrace this.
- Lastly, there is another saying from a former colleague of mine, Anil Podduturi, and he used to say that everyone needs to “control their narrative.” By narrative, he meant story. A good PM controls the narrative for the team, both inside and outside the company. Why are they working on what they are working on? How is it going? What do they need to make progress?
I’ve had teams where the rest of the company was scratching their heads, confused about what a team was doing and why they appeared to be accomplishing little. Often, the outsiders draw unfair conclusions, but if the PM doesn’t control the story, people come up with their own stories. The same thing goes for explaining to your customers why you are making a change. If you don’t, they will come up with their own angle, and often it won’t be complimentary.
Frankly no PM is better than a bad PM, because then the eng lead or designer steps up. However, I’d say the same thing about a bad designer or engineer as well.
What are the ingredients for a strong product organization (prod-eng-design)?
First, I want to talk about shared goals. Do not, not, not create separate OKRs. I’m not talking about maintenance items like keeping 99.9% uptime. I’m talking about goals — the things we are chasing.
- What happens when people set goals? They chase them. What happens when engineering is chasing a goal of increased test coverage, a faster or safer release process, or a move from an outdated infrastructure to a better one — while product has been tasked with business goals like increase engagement hours or solve a retention problem or ship this feature so that we can open up a new market? We’ll be at odds, that’s what. Which is stupid. This creates conflict.
- We can’t do anything effective without each other. And you might say, ha no, if we have designers and engineers, we can do just fine, but I would say: if you’re shipping something good and impactful, someone is playing the role of PM even if that’s not their title
- And since we can’t do anything good without each other, it’s stupid to be at cross purposes, ever. Instead, we need to widen our perspective on the kinds of business value we need to create and design the right compromises, the right priorities. Uptime, performance, release time, test coverage — these are business problems that are as much the PMs’ problem as the engineers.
This takes getting together at the leadership level, throwing all the goals and challenges on the table, and working it out together. For example, at Meetup, our code maintenance “surface area” was 5-10 times bigger than the code we were working on at any point in time to deliver new customer value. Our teams had ambitious OKRs trying to create new value for customers and the business, but they also had maintenance responsibilities. When a maintenance issue arose, it derailed the team from their goals. Something had to give somewhere. This wasn’t Yvette’s problem (our CTO) or my problem (as CPO) — we had to solve it together. We needed to pick the least worst option of how to balance these competing issues, and then together sell our CEO on our path forward.
Second, help create strong guardrails for the teams. This means a strong and well-understood company strategy, which allows you to push decision making down and out to the edges, and coherent working principles and values. And engineering, design and PM can’t just focus on their own culture to the exclusion of the others, but work to build understanding, appreciation and empathy.
Oh and one more opinion on this topic — as a general rule, I believe in escalation paths not giving decision-making “ownership” to a specific person. Everyone on a team is entitled to ask questions and challenge a decision. If there is a fundamental disagreement on a team which they can’t resolve, they should escalate it up a level — not for the next level to make the decision necessarily, but for that level to help them make the decision. At Meetup, I said, if your team is stuck, you have to pull in a VP – any VP in product, engineering, or design. I trusted all of the VPs to look at the bigger picture and help the team make a small call.
All this starts at the top. If the executives walk the talk, people will follow. If there is tension between the leaders, people know and the teams will be wary of each other.
A good CPO and a good CTO will make better decisions together. They will see each other’s blind spots. The real issue, as an exec, is whether you have a quality partner whom you can trust. Do you have the kind of relationship where you can push each other while respecting and appreciating each other’s strengths?
I’ve run both product and engineering multiple times in my career, and have known engineering leaders who could successfully run product and design. There are times when it’s useful to have everything report to one person, but I would always rather have a strong partner. Granted, I would rather have no partner than a weak partner, but the best is a strong partner.
How can engineering help?
I’ve got 4 initial responses to that:
- Ensure that the engineering lead on the team treats the PM-Design-Eng triad as their “first team” (and not the rest of engineers)
- Hire a diversity of engineers. You need some who will see an idea and worry about implementation risks, but you also need some who see that and get excited about the challenge. And very importantly, you need some who can be creative partners to the PM/designer, which means having an interest and making time for customer research, business strategy, brainstorming, etc.
- Ask your eng leaders to take PMs under their wing — expose them to engineering problems, values, the “why” behind your processes and all the things you’re trying to do.
- Train engineers on two concepts which do not come naturally:
- First, don’t try to build the perfect system. There is no such thing. We’re constantly rebuilding our systems every few years. Accept it. (No, I’m not saying build crap) This ties into the need for everyone to stay aware of which decisions (business, technical, whatever) are one-way doors versus two-way doors.
- Second, and this one is really tactical but I swear it’s one of the biggest killers of team pace, if you get confused about what you are building and why, stop coding and ask. And you would think that would slow down the team, breaking flow and all that, but the real time and energy killer is unwinding all the work an engineer did when they made the wrong assumptions. It’s irritating to break flow, but necessary.
How should a good, modern PM organization think about metrics in 2021?
First, I do have to get the obvious out of the way. Use data. Instrument the product. Have the ability to split test. Have a strong partnership with data science.
That said, I do warn PMs not to get lost in a sea of data. They should have a bunch of metrics they watch, but only a few they really prioritize. More data does not magically mean more answers. First off, data is backwards looking. Second, there is bias and gaps built into the data.
I fear a growing theory that we can just throw things into reinforcement learning algorithms and with enough data, we’ll get perfect prediction engines. I don’t believe it. First off, without qualitative research, you can’t separate causation from correlation. And you can say, oh but that will smooth out over time and with enough data. To that I say the world is too complex and changes too fast.
What will happen when we’re all using AI prediction engines to make decisions? Our systems, and thus our behavior, is going to feed off of each other, and we’re going to have unintended consequences, just like when they introduced automated trading systems and realized they had invented new ways to crash the market. I’m not saying don’t use ML. I’m simply saying, let’s not put it on a pedestal. But then again, humans are always looking for oracles. It’s in our nature.
I’m also constantly trying to get my PMs to connect the dots between product metrics and financial metrics, but not just financials — the actual value of the business. Not all ROI is equal. (Note: blog post coming on this)
(If time to discuss): I think our field has gotten overly obsessed with engagement at the expense of customer value.
Final note
The actual conversation didn’t go exactly like the above, but I hit on a lot of those points. Since this was on Zoom, while I was engaged with the interviewer, there was an interesting back-channel going on in the chat. I’m going to do a follow-up post touching upon some of the questions or points of confusion I spotted later when I looked at the chat log.
Top image from Kelly Sikkema on Unsplash