I give both Mark Suster and Chris Dixon credit for pointing to elephants in the room (hot air balloons, etc) of our industry –- topics that most entrepreneurs feel they can’t talk about in public without risk of getting burned.
Suster’s latest post on statistics and research firms is no exception. Go read it. When Jason Calacanis began his tirade on Comscore and “pay to play”, my brain couldn’t help but go to the research firms. I wasn’t willing to blog about it then, but what the hell, here are some thoughts:
- CYA. Big companies are so desperate for a self-proclaimed “guru” to cover their asses and provide desperately sought direction that they swallow whatever they are given.
- “Gimme a number, any number”. Bankers and management consultants keep the game going, because they need external, supposedly-objective, statistics to help drive strategic suggestions.
- Just the “facts”. The Press is so desperate for concrete numbers (the more exciting the better) that they’ll reprint whatever is handed to them.
- Payola. The analyst firms are businesses in search of both revenue, which leads to a bias towards paying clients, and media attention, which leads to the creation of reports and headlines that draw attention.
- How big? VCs propagate this nonsense by asking for proof on un-provable things.
- Do what you gotta do. Throughout it all, entrepreneurs hold their nose and play the game, trying to educate the analysts as best they can, paying when they need to.
Create your own truths
While we’re talking about bullshit statistics, at my first job out of college, I learned how easily the press could be manipulated by an ethically-flexible entrepreneur. Journalists are so desperate for hard numbers that they’ll run with whatever you tell them, as long as it is stated with great confidence and wrapped in a cover of objectivity. They might even refuse to write an article unless you have hard numbers, increasing the incentive to make shit up.
The next journalist to pick up the story will read the first article and, since they’re pounding out an article at high speed under deadline, will take the statistic without any further due diligence. All of a sudden you have multiple media sources reporting a statistic. It’s no longer your number, it is an objective fact! You’ve now got the stat you need to justify your ROI to prospects. It’s not pretty, but it happens.
Painting with a broad brush
I know I am making huge generalizations here that cannot be applied to every individual in the research space. I’ve come across some really diligent analysts who care a lot about their space and do their best to make educated, fair guesses. Unfortunately, I’ve also run across the opposite and, in my enterprise software days, have definitely felt the pressure to pay to get good treatment. I’m just not sure what can be done about all this because humans have a need for false certainty, and there will always be self-proclaimed gurus ready to fulfill that demand.