All Talk, No #Measure

Hypocrisy in Web Analytics?

Before every eMetrics (I’ll be in San Fran teaching Basecamp, at the Gala, etc.), I try to ask myself, what is the most critical issue facing the web analyst community right now?  Then, at the show, I ask everyone I run into what they think about this issue.

There’s lots of issues to choose from.  Career path I think is a big area of discussion, given the mergers in the space and trend towards outsourcing.  Then there’s the “we don’t get no respect” thing; senior management doesn’t seem to listen / understand / act on the information provided.  And one of my favorites from the past is still out there, data torture – people being pressured to manipulate data to reach a predetermined analytical outcome.

But seems to me, more important at this juncture is trying to resolve why there is so much written about the importance of “the customer” but very little measurement at the customer level.  Think about it.  Customer experience, customer centricity, the entire social thing, it’s all about customers.

But when folks wants to trot out “proof” that this or that approach is the road to the promised land, they analyze impressions, visits, clicks, etc.  Visitor-level stuff.  Does that seem like the correct approach to you?  Seems to me, if you want to provide knowledge about customers, you should measure customers.

One thing we know is customers do express behaviors through a web interface that are not relevant to the future behavior and value of the customer.  One of the earliest and most widely publicized incidents of this was with Amazon gift purchases.  People went on and on about buying a gift from Amazon unrelated to their interests yet having that category Marketed to them relentlessly over time, even though they never purchased from the category again.

This problem was eventually solved by Amazon using Recency, a classic customer behavior metric – only more Recent behavior was used to make suggestions.  Recency is predictive; and lack of behavior is often just as important, if not  more important, than expressed behavior when trying to understand customers.  Unfortunately, most web analysts are trained to look for expressed behavior, not the lack of behavior.

Further, just because an event of some kind happens in the stream of web activity does not mean the event had any affect on the behavior of the customer.  Display impressions, searches, social interactions, all of it – how can you tell whether the event had any effect on the customer at all?  The only way is to measure at the customer level, for example, comparing the behavior of customers who were exposed to the events with customers who were not exposed.  Or, modeling different mixes of events against customer behavior over time, a “marketing mix” model of sorts, to stretch the idea.

Now some people are going to say. “But Jim, we don’t have web tool access to this data!” or “We don’t pass web data to the back end” and all manner of other related excuses, to which I would say,

“Where is your curiosity?”

Clearly, a unified database is best.  But just because your company can’t afford an advanced WA tool doesn’t mean you can’t do this.

I mean seriously, get a dump from the order management system into a spreadsheet.  Run a query against the CRM database.  Look up individual cases in the customer service or lead management systems.  This the way analysts make breakthroughs, how  business cases are built.  If key web data (campaign codes, logins, etc.) doesn’t make it into the back end, why?  If form data crosses over, how hard could it be to send a campaign code, login, or other critical data?  With proof, then pitch the advanced WA tool, or systems, processes, people, whatever you need to make it easier to analyze customer level data.

OK, so let’s hear all the reasons why it’s fine to draw customer-level conclusions using visit-level data, or why you can’t do the above, which I’m sure will include some of the following:

1.  My boss doesn’t care about customer-level data, ignorance is bliss, pseudo-analysis is OK

2.  I’m too busy learning very little about a lot of things instead of going deep on the most important stuff

3.  Shiny objects rule, so see #2 above

4.  I’m a web analyst, back-end data is not my thing

Other reasons?  What do you think?

Do you see the hypocrisy in claiming to understand customer behavior based on visit behavior?

Let’s talk about this at eMetrics San Fran…and Toronto too.

6 thoughts on “All Talk, No #Measure

  1. You’re highlighting an important point.

    In my mind, there’s an activity stream, which on the website is largely anonymous non-PII data. We attempt to get at identity level data using the HTTP Cookie. And fail. And then lie to ourselves about the failure and the original lie.

    There’s customer level of data, which is not anonymous, but typically sealed away in a CRM system. Without an activity stream.

    And then there are loads of other bits of business performance data.

    Web Analytics does usability and aggregate channel performance somewhat well. The expanded set of web analytics techniques, especially relating to aggregate behavior and pre-click analysis, are especially robust. But the data is not sortable at a PII level. We know what the herd is doing. We don’t who the lead buffalo is.

    I’m not making an excuse. The integration of CRM-WA could be major. We’re 17 years on and it’s only really happened at a handful of companies. And their Web Analysts are actually called Business Intelligence (BI) people. And it’s a beautiful thing.

    But no. I don’t believe that I can draw customer level inferences based on anonymous activity streams.

    It’s about picking the right tool for the right goal.

  2. Good points.

    I’m not looking for a miracle, 360 degree view, etc. More like hacks that tell the customer story.

    Example: PPC campaign code: high conversion rate vs. low conversion rate: how do the return rates / 1x buyer stats for each campaign compare? Because that tells you what you can really spend on the click, from CFO perspective

    Example: social link ID: can decelerating social behavior predict customer defection? Can be used as “win back” trigger? (yes to all)

    Simple but financially / strategically really important ideas. The most financial leverage is at the customer level, not the traffic level.

  3. Great post! Totally agree. We did an analysis for a highly-considered purchase site that ended up with profiles for several kinds of visits, sequenced as visitors got further into the process. What a revelation! Totally different from the high-level picture of site activity.

    This is not quite what you’re talking about, Jim, but I just want to jump in and endorse the idea of understanding “the visitor” over time.

  4. Hi Chris, I think your point is directly in line with what I’m talking about.

    Put a different way, I wonder why are marketing person would not want to take credit for at least part of the future stream of sales from a new customer acquisition, say first 3 months. That’s a customer view, as opposed to the visit view.

  5. Hi, Jim,

    I totally agree – in fact, I agree, because I work with my clients to do precisely the analytics you describe, in datasets like you describe.

    One of my breakouts at the recent Adobe Omniture Summit was about how easy it is, when you really set out to do it, to build a dataset (in our case, in Adobe Insight) of data at the “higher” level of Customer, and key a bunch of datasources (Web included, obviously) on the Customer.

    Let me answer your four main points/objections really quickly with what I know or see from my own clients (who, I’ll admit, are rare in the WA world, because, admittedly, they’re using Insight, which is a solution designed and offered PRECISELY for this type of versatile data schema and these type of analyses):

    1. My boss doesn’t care about customer-level data, ignorance is bliss, pseudo-analysis is OK

    The most successful clients I have are fortunate to have a manager with the vision to see that by caring about the customer-level data, they bring more valuable information to the table and impact the business more solidly over a longer period of time. They also help cut more, real inefficiency out of the online marketing in many areas.

    2. I’m too busy learning very little about a lot of things instead of going deep on the most important stuff

    The most successful clients I work with have the vision to realize that focusing on one or two key questions at a time allows them to give deeper, more well-reasoned answers back to the business.

    3. Shiny objects rule, so see #2 above

    Wow… the dollar is the second-most-shiny object, so I’d say that optimizing at the customer level can help toward the end of getting to a great “shiny object” more quickly. And in my opinion, in this economy, happy, long-term productive customers are THE most shiny object, and only by focusing on the customer level can we understand the richness of our customers and their interaction with us as organizations.

    4. I’m a web analyst, back-end data is not my thing

    I’ll buy that one for now, but it won’t keep anyone who utters it in work very long in the future. Keep the blade sharp, right?

    I’m pretty sure you already know and realize all of my responses above, but I certainly don’t want to put words into your mouth, and just wanted to throw them out there as my own experiences and opinions – and the ideal that I try to push clients toward as “best practice.”

    We certainly have a long way to go, but there are many organizations that are “getting it” and “getting there” with their data.

    In fact, I’m working with a few clients right now whose entire dataset in Insight is Customer-centric – they even throw out all of the data for the Visitors who haven’t been tied back up to a Customer.

    I’m particularly thrilled when I see clients really start to “see” the value as they look at their customers as customers from their interaction in various online channels…. then add the offline (“wow, did all of those customers really book a reservation but never show up?”)… then add other channels (“gosh, that customer buys a lot, but ties up a ton of time complaining in the call center”, or “thanks, customer, for all the great stuff you said and shared about us after your recent visit…”)

    The sky is the limit. Thanks for pushing more of us there.

  6. Michael, that’s fantastic, thanks so much for the considered reply. I take it the above rules out the “no tools available to do this” excuse…and in the same breath, to be fair, say Omniture is not the only WA tool group with these capabilities.

    I am really curious about the state of #1. I would feel much more comfortable knowing there are WA folks out there who *want* to do this, want to learn how to cross to the customer level. But they just don’t because their boss is not curious and not interested in “truth”.

    Far more disappointing would be knowing the bosses out there would be interested, but the analysts are not curious enough to explore the options.

    And as I said above, I don’t buy the “tool cost” thing as a barrier. The process I talked about is how you make the case for an advanced tool – find the data in the backend, make the correlations to customer / profit, show the shiny dollar benefit, then get the tool that allows you to do same, but faster and deeper.

    I just hope we are reaching folks with these possibilities, and thanks for adding to the list of voices saying “people are doing this, right now”.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.