Fear and Loathing in WA
You may recall I wrote last year about the explicit or implicit pressure put on Analysts to “torture the data” into analysis with a favorable outcome. In a piece called Analyze, Not Justify, I described how by my count, about 50% or so of the analysts in a large conference room admitted to receiving this kind of pressure at one time or another.
Since then, I have been on somewhat of a personal mission to try to unearth more about this situation. And it seems like the problem is getting worse, not better.
I have a theory about why this situation might be worsening.
Companies that were early to adopt web analytics were likely to already have a proper analytical culture. You can’t put pressure on an analyst to torture data in a company with this kind of culture – the analyst simply will not sit still for it. The incident will be reported to senior management, and the source of “pressure” fired. That’s all there is to it.
However, what we could be seeing now is this: as #measure adoption expands, we find the tools in more companies lacking a proper analytical culture, so the incidents of pressure to torture begin to expand. And not just pressure to torture, but pressure to conceal, as I heard from several web analysts recently.
One bright young analyst went “beyond the call of duty” on his analytical project. The analyst gathered relevant data not just from the WA tool, but from Finance, Customer Service – all around the company. The report painted a detailed picture of cost to acquire customers through various methods and campaigns, and was presented to the head of Marketing – also the analyst’s boss.
The analyst was told under no circumstances was this report to ever be produced again. Further, the analyst was told to destroy any “evidence” this project / report ever existed. And finally, the analyst would now be required to send all analysis through the boss first before anybody else sees it.
That’s shameful behavior for an exec. And apparently, this kind of thing is happening more and more often. I’ve heard plenty of “if we want your opinion, we’ll ask for it” stories, but this is the first time I’ve heard so many stories about concealing results.
Here’s a scary thought: what if the stories about web analytics not driving business value are primarily concealment stories? What if the tool / analysts actually did provide value, which was then hidden from Senior Management?
My concern about this issue is wider than screwed up company culture and management. What I’m more concerned about is screwed up people, analysts who may come to think this kind of behavior is normal and just part of being an analyst.
This matters because as this new generation of analysts moves to other companies and throughout the ecosystem, these pressure to torture situations could become “accepted” and even spread as “part of the game”.
It is never, ever OK to manipulate or hide the results of an analysis. It’s not part of the job. The role of an analyst is to analyze, not justify or conceal bad news.
Now, I realize some folks are thinking, “Yea, that’s great Jim, I’ll just get myself fired by being an analytical hero”.
I’m not saying you should respond to data torture pressure by falling on your analytical sword. What I am saying is you – and management – need to know this kind of pressure from a superior is shameful, not a “normal” part of being an analyst. And as soon as you can, you should get a job somewhere people respect your professional opinions. Don’t have to agree; but must respect.
Like the company you work for? Ask a buddy in Finance if they could use a web analyst. Pretty sure Finance would be interested in fully-loaded cost to acquire new customers by source!
What really troubles me about this situation is it’s rarely ever talked about, so could be worse than people might think. At the very least, Senior Management should know about the potential for this to happen and lay down some rules. Perhaps even seek some cultural guidance on this topic (here’s a start – Fear of Analytics).
So, I want to put this message out there, perhaps create a resource for people who are looking for information on this topic. It would be great to have examples so managers can understand and be on the lookout for these situations. Plus, I’m sure there are some terrific stories out there about either giving in to the torture pressure or resisting it!
What about you? Were you ever pressured to torture the data? What happened? Did you comply? How did things come out? Tell us with a Comment. Feel free to post anonymously, leave out company names.
