Does this issue affect the Web Analytics Maturity Model?
A conference call with a Potential Client last week jogged my memory on a couple of events that happened during the flurry of Web Analytics conferences this Spring. Here’s a portion of the call…
PC: “We’ve tried proving the profitability of our Marketing efforts and can’t seem to get the numbers working correctly. So Jim, what we’d like you to do is take all this data we have, and justify the Marketing decisions we’ve made by proving out the ROI.”
Jim: “I’m sorry, did you say justify? To me, justify means “find a way to prove it works”. Is that what you are asking me to do? Wouldn’t it be more beneficial to analyze the results, and then optimize your Marketing based on these results?”
PC: “Jim, around here we’re pretty clear our Marketing works, and Management knows this. But Finance is asking for some backup, some numbers to justify the spend, not to analyze it. We don’t need analysis, we need your ‘expert credibility’ to help us out with this.”
Jim: “I see,” thinking this is not a job I’m going to enjoy. It’s the old ‘buy an outside expert’ routine, which I detest.
PC: “Jim, the team is united behind this mission, are you on board?”
Jim: “Well, perhaps I could be on board, as long as what you want is an analysis, which may also justify the decisions you have made. But it might not, so I just want to be clear on what…”
PC: “You know what Jim? I don’t feel we’re going to have a fit here, I’m getting you’re not a team player. Thanks for your time”. CLICK
Sigh. I’m actually grateful they hung up, I really dislike explaining to people why I won’t work with them.
And that’s when I remember one of the most interesting moments for me during the WA conference season happened at webtrends Engage.
I was on the “Socialization of Data” panel with a great crew composed of @jacqueswarren, @anilbatra, @johnlovett, and @bgassman. We were talking about what web analytics might look like organizationally in the future. Specifically, we were discussing the “Center of Excellence” concept, where all the senior analysts report to the same person, and this person typically reports directly to the C-Level.
There are many reasons this idea is a good one, but the one I often stress is relieving the pressure, explicit or implicit, on an analyst to produce a certain result from their work. In other words, to “justify” a program rather than analyze it and get at the truth.
I said something like, “You really don’t want analysts reporting organizationally to the group they are responsible for analyzing. This set-up tends to create a lot of pressure on the analyst to prove a program is working by torturing the data to get a desired result.”
And about half the heads in this good-sized auditorium bobbed “Yes”.
These are the people who have been asked to change a date range, to modify a filter, to exclude a segment. To justify a program by torturing the data into saying what someone wants it to say.
That’s sad. Really sad. In fact, it’s downright poisonous to the long-term health of Web Analytics (or any other analytical discipline) as a profession. It’s a rot from within, difficult to cure.
The existence of “justification” means the business is really not being run by the numbers. What it means is the business continues to be run by “gut feel”, and the numbers are used to justify on the backend.
That’s not analytics, that’s a lie.
Depending on your experience, if you work in an environment like this, you might want to look elsewhere for a job, because eventually this game collapses. It has to, you see; other people can and will get the correct numbers.
Especially in Finance.
And speaking of Finance, here’s the second most remarkable thing that happened to me on the Spring Tour. I was talking with this web analyst who reported into Marketing. One day, the CFO said to him, “You know what? I think you should work directly for me. What do you think of that idea?”
After an initial heart attack, the analyst said OK. And he is so much happier, giddy in fact. He loves his job again, really is fired up to get to the desk in the morning. Why the change?
“The people in Finance get it, they understand what I have to say. Nobody asks me to fudge the data in Finance, ever. They just want the truth – good, bad, or otherwise.”
Funny how that works; I know exactly what he means. If you are a profit-driven Marketer – and you can prove it – the CFO is truly your best friend. Because a CFO gets Profit Math.
Then Web Analytics Maturity Models started getting a lot of attention due to the release of a new one from webtrends. So I’m looking through the model and suddenly these two experiences from the Spring tour above pop into my mind. I think:
You can have the best processes and procedures on the planet, but if you also have this Justification thing going on, if your analytical org chart is designed to fail, these Maturity Models are all just crap. Literally. A gigantic waste of time for everybody.
Worse than Scrap Learning.
Now, I’m not picking on webtrends here, because I like their model. But what I don’t see in this model or the others is anything about properly Managing Analytical Cultures, like these org chart conflicts that drive Justification. Not sure if this issue belongs in Governance, or Domain Expertise, or some other place. I know the IT side has established “formulas” for Maturity Models, so maybe this org chart stuff doesn’t belong in the Maturity Model itself.
But this issue of reporting structure needs to at least be addressed in Maturity Model supporting documentation. What good is it to have all the gears turning properly if the analysis itself is faulty, and drives continued poor decision making? What kind of Maturity is that?
Speaking of chaos, if you’re in web analytics and find out your area has been targeted for downsizing but you would like to stay with the company, here’s an idea. Head down to Finance and ask them if they would like their own web analytics person. You might be surprised at the response. After all, what is it most of the people in Finance do?
That’s right, Analysis.
What do you think about this issue? Have you ever been forced to Justify? Are you asked to run reports with “special parameters” for some programs? To bury or exclude certain reports?
Got any good data torture stories?
Does this organizational topic belong in the WA Maturity Model? If not, how would you handle it, where does it belong?