People love metrics. I love metrics. I love getting arguments out of opinion and into data. However, I’m also deeply suspicious of metrics, especially since I focus on new products and thus live in a world of constant change and evolution.
Some points of discussion:
New Employees
I have new employees join and ask “what targets do I need to hit to be successful?”
There are two sides to this. First, it is important to set outcome goals so that everyone is on the same page regarding expectations. Second, if you are doing anything entrepreneurial, chances are high that the performance metrics you set on Day N will be wrong at N+60, or even sooner.
I try to make a pact with the employee: if at any point, you or I realize that you’re working on the wrong thing, let’s agree to blow up the metrics and rethink. I also have a goal of reviewing the metrics, not just the results, every three months.
The same thing should go for any team. Set goals. Set checkpoints when you evaluate them. Be willing to blow them up at any point in time.
Choose wisely
You want to make sure you are paying attention to the right metrics. Obvious, right? But it’s easy to fall off the rails here. We recently launched a new business line for a client, and now have revenue coming through the door. Before the project was funded, we created a financial model that forecasted customer growth and how that might translate to revenue and profit. Once the product was live, some people started to fixate on hitting the “new acquired customer” target for each month from the original model.
But while new customers was an interesting metric, it was not the true goal. Our true goal wasn’t to have a certain number of new customers, but rather to hit certain revenue targets. We could hit our revenue goals in a myriad of ways — acquiring new customers, raising prices, reducing churn, and more. Fixating on a metric that fed into our goal, rather than our true goal, could handcuff our thinking. A young business needs flexiblity not handcuffs.
Locking Metrics in Stone
One of my current management headaches is tied to the M&A (merger and acquisition) activity that preceded my taking the helm. The company wanted to pay for performance, and so it created earnout structures and locked this into the acquisition legal agreements.
Unfortunately, this only works if you can reasonably predict how operations will unfold.
Our company, even though it is a services business and not completely re-inventing the wheel, is trying to push the edges of how consulting companies operate. We are testing out new models. Rather than improving performance, the earnout structures have turned out to really constrain us in all sorts of ways because M&A legal agreements are much harder to rip up and rethink compared to a set of employee goals.
Interpretation
Metrics are only as good as how you intepret them. Some are clear cut, but in the early days of a product, you often don’t know what “good” vs “bad” performance really is.
The Internet is awash with information, but it is not awash with comparables. Take conversion rates, for example. In one project, we are closely looking at our new customer conversion rates. We know if we are making progress, which is powerful, but how do we know if something is good? What do we have to compare it to, other than our previous numbers, our experiences on other projects, and perhaps some whispered shared metrics from peers at other companies.
The intepretation challenge doesn’t stop just with lack of comparables. People can also slice up data in ways to confirm or deny their own opinions. Once upon a time, I did M&A and IPO investment banking for tech companies. We always laughed that every single bank was “#1 in tech”. Everyone just sliced the numbers in a way that made them come up as number one. Every claim was true — everyone was number one at SOMETHING.
Final Thoughts
I like that the startup world is trying to rely on metrics more than pure gut alone. I just think that some people are a bit too caught up with the idea that we need to measure everything, and let metrics rule all.
There needs to be room for judgement and flexibility.