Hypocrisy in Web Analytics?
Before every eMetrics (I’ll be in San Fran teaching Basecamp, at the Gala, etc.), I try to ask myself, what is the most critical issue facing the web analyst community right now? Then, at the show, I ask everyone I run into what they think about this issue.
There’s lots of issues to choose from. Career path I think is a big area of discussion, given the mergers in the space and trend towards outsourcing. Then there’s the “we don’t get no respect” thing; senior management doesn’t seem to listen / understand / act on the information provided. And one of my favorites from the past is still out there, data torture - people being pressured to manipulate data to reach a predetermined analytical outcome.
But seems to me, more important at this juncture is trying to resolve why there is so much written about the importance of “the customer” but very little measurement at the customer level. Think about it. Customer experience, customer centricity, the entire social thing, it’s all about customers.
But when folks wants to trot out “proof” that this or that approach is the road to the promised land, they analyze impressions, visits, clicks, etc. Visitor-level stuff. Does that seem like the correct approach to you? Seems to me, if you want to provide knowledge about customers, you should measure customers.
One thing we know is customers do express behaviors through a web interface that are not relevant to the future behavior and value of the customer. One of the earliest and most widely publicized incidents of this was with Amazon gift purchases. People went on and on about buying a gift from Amazon unrelated to their interests yet having that category Marketed to them relentlessly over time, even though they never purchased from the category again.
This problem was eventually solved by Amazon using Recency, a classic customer behavior metric – only more Recent behavior was used to make suggestions. Recency is predictive; and lack of behavior is often just as important, if not more important, than expressed behavior when trying to understand customers. Unfortunately, most web analysts are trained to look for expressed behavior, not the lack of behavior.
Further, just because an event of some kind happens in the stream of web activity does not mean the event had any affect on the behavior of the customer. Display impressions, searches, social interactions, all of it – how can you tell whether the event had any effect on the customer at all? The only way is to measure at the customer level, for example, comparing the behavior of customers who were exposed to the events with customers who were not exposed. Or, modeling different mixes of events against customer behavior over time, a “marketing mix” model of sorts, to stretch the idea.
Now some people are going to say. “But Jim, we don’t have web tool access to this data!” or “We don’t pass web data to the back end” and all manner of other related excuses, to which I would say,
“Where is your curiosity?”
Clearly, a unified database is best. But just because your company can’t afford an advanced WA tool doesn’t mean you can’t do this.
I mean seriously, get a dump from the order management system into a spreadsheet. Run a query against the CRM database. Look up individual cases in the customer service or lead management systems. This the way analysts make breakthroughs, how business cases are built. If key web data (campaign codes, logins, etc.) doesn’t make it into the back end, why? If form data crosses over, how hard could it be to send a campaign code, login, or other critical data? With proof, then pitch the advanced WA tool, or systems, processes, people, whatever you need to make it easier to analyze customer level data.
OK, so let’s hear all the reasons why it’s fine to draw customer-level conclusions using visit-level data, or why you can’t do the above, which I’m sure will include some of the following:
1. My boss doesn’t care about customer-level data, ignorance is bliss, pseudo-analysis is OK
2. I’m too busy learning very little about a lot of things instead of going deep on the most important stuff
3. Shiny objects rule, so see #2 above
4. I’m a web analyst, back-end data is not my thing
Other reasons? What do you think?
Do you see the hypocrisy in claiming to understand customer behavior based on visit behavior?
Let’s talk about this at eMetrics San Fran…and Toronto too.