WebTrends Score

Is “Engagement” Physical or Emotional?

From the WebTrends Press release:

“WebTrends Score is a patented technology solution that evaluates visitors’ online behavior by quantifiably measuring the level of engagement or interest they have in content, products and services. By establishing rules that assign values to specific visit and visitor activities, marketers can go beyond conversion to evaluate the success of their efforts using realized and potential customer value”.  Bold in the last sentence is mine.

Two value dimensions – “realized”, which is history, and “potential”, which is predicted, future value.

This is great work, because now we have a more accessible way to test what is important to the execution of High ROI web site efforts – historical data or predictive data.  And I’m wiling to bet anybody the answer will be the same as database marketing folks have been discovering for years: predictive data.  The leverage you gain from prediction far outstrips the leverage you gain from understanding the past.  Once you have a prediction, you can then inform this prediction with a historical view, providing context for the execution against the prediction.

To many folks, Score will be an unbelievable geekfest of historical tracking capability.  “Look at all the ways we can assign value to visitors, create scores, rank them, trigger content based on the scores”, etc.  Sure, a much easier to work with execution of the basic Content Groups idea; advanced Frequency analysis.  That’s important; easier is always good.  Broader applicability to all kinds of events is good too.  But they’re still events, and they are still history.

Can I ask you something?  Why have web analytics folks over the years always thought it was important to segment out New from Returning visitors?  Right, because the behavior of Returning visitors is often quite different from New visitors.  And it’s worth knowing this, because Returning visitors are good, right?  If they are coming back, they must be happy with the site, “engaged” with the site, don’t you know.  So it follows these visitors have higher value and should be tracked as a unique segment, because they are worth paying attention to.  The have high historical value from repeated visits, and have an implied likelihood to come back, since they are repeat visitors who have value in the future.

This is a prediction.  A Repeat visitor is more likely to come back to the site than a New visitor.  Simple.

So, let’s take a New Visitor who interacts with a wide variety of site components, or spends a long time on the site, or both.  Compare this visitor with a Returning Visitor who interacts with the same wide variety of site components, or spends the same time on the site, or both.  Which is more valuable to the company, do you think?

Both visitors have the same “realized” or historical value.  If you stop there with the analysis, you don’t have anything actionable.  But if you toss in that one visitor is New and the other is Returning, all of a sudden you have an actionable difference.  One has higher “potential” value, value in the future, because a Returning visitor is more likely to come back than a New visitor.  Given a dollar to spend, and betting on where you would get the highest ROI, which segment would you invest in given they have the same behavior on the site, the New visitor or Returning visitor?

The answer is this: it depends on whether you care about building a business with legs under it. 

If you just want to churn through New visitors and don’t care if they come back, you choose option 1, and invest in the New visitor.  You don’t change your marketing, content, or navigation to create satisfaction and repeat visits.  You focus on the physical engagement with the site.  Plenty of examples around of how that works out in the end, though nobody seems to think that will ever happen to their site. 

Or you invest in the Returning visitor who is building the business, who is providing a forward revenue stream after the first visit, who is suggesting the site to others, blogging about it, etc.  The visitor who is truly engaged with the site on an emotional level.

Here’s a suggestion: before we spend years creating really elegant reporting on history only to realize that history is the 2nd most useful dimension of the 2 value dimensions provided in WebTrends Score, I would like to remind folks of a couple of things:

1.  Just because I thrash around your site and interact with a lot of elements, doesn’t mean I am happy with your site; indeed it could mean the opposite – I can’t find what I am looking for, I hate the interface, etc.  This “thrash problem” is in fact the same argument often used against duration as a measure of engagement, “just because I spent a long time on the site doesn’t mean I am engaged”, for some of the same reasons above and others. 

So why don’t we just agree that neither “time spent” nor “elements visited” is really a good measure of engagement?  That “history” is not really relevant to engagement?  It’s very relevant to the value of the visitor, it’s “realized” value, value in the past.  But by itself, it’s not very predictive of anything.  At best it says “visitor used to be engaged”.

After all, it’s history.  And history means “used to be”.

Put another way, I have 2 visitors who were both very active on the site, interacting with all the cool features.  Last visit of one was yesterday, last visit of the other was 6 months ago.  Which visitor is more engaged, has higher potential value to the company in the future?

2.  Just remember that you can’t trigger Score-based profiles on a visitor who doesn’t come back.  This has always been the soft underbelly of on-site personalization; it’s a complete waste of resources customizing all kinds of trigger-based scenarios for visitors who never come back.  So if you can predict the likelihood of a return visit, you can optimize the system to address visitors with the highest potential value.

It’s true “visitor engagement is not something you can measure using only a stopwatch“.  But it’s also true that you can’t measure engagement based on number or breadth of events.  That’s a very shallow view of engagement, unless you are in a business where you simply don’t care if visitors ever come back.

If you have such a business model, WebTrends Score allows you to base your engagement metrics on historical event Frequency, the physical interaction with the site.  For everyone else, Score also allows you to base your engagement metrics on potential / emotional value, a prediction of the value of the visitor to the business in the future, and then execute strategy based on the historical context (Frequency) of the visitor.

It’s good there is a choice, as it’s clear some sites will prefer historical “realized” value over potential value as the primary measure of engagement.  These sites will embrace viewing history as more important than predicting the future.

Personally, I find management folks to be more interested in understanding the future than understanding the past.

What do you think?  If the above makes sense to you, let me know.  If you think it’s irrelevant or misguided, tell me why.

Bonus: If you are dealing with multi-channel or multi-system customer analysis, tell me how much easier it would be if you could summarize the value of the web component with these two variables – realized value and potential value – then represent the web value of a customer with a 2 digit score in the offline customer record? 

How many of you multi-database folks are already doing something like this with the web data?  C’mon, give it up…we won’t tell!

Will Work for Data

But will do a sub-optimal job…

Trying to catch up on what is going on in the analytics blogosphere, and it seems like I’m seeing a common thread – we’re getting much better at analyzing customer data, but whoever is in charge of Turning Customer Data into Profits is not quite with the program yet. 

Based on my experience, and assuming the people responsible are Marketing folks, the challenge to solving this problem often lies in understanding the difference between executing against behavioral data and executing against data about “characteristics” like demographics.

Marketing is not always about buying mass media, yet most Marketing people have never had to create and execute a campaign using behavioral data against a behavioral Objective.  So they do what they have always done – they create campaigns based on characteristics – and then execute against behavioral objectives using behavioral data.

This is a recipe for sub-optimal performance.  It’s like buying a car with a high performance engine then putting the cheapest gas in it you can find and never getting a tune up.  Sure, the car will run, but it’s not going to run very well, and you sure are not going to win any races with the competition.  Provided, of course, they don’t treat their car the same way.

For example, Ron is commenting on weak segmentation practices and lack of understanding the new customer experience in banking.  He is absolutely right.  Segmenting by “number of products” is often a static characteristic; segmenting by “change in number of products” is behavioral and many times more profitable.  As for new customer experience, the initial experience defines a customer’s “view” of the company and I don’t think I have to explain the importance of that.

Kevin is bemoaning the lack of temporal segmentation and use of appropriate creative for this segmentation by many e-mail folks.  He is absolutely right.  You want to speak to the customer based on their level of engagement with the company, not in terms of static perceptions.

Avinash perceives a problem coming down the road with behavioral targeting, that is, while the machine is smart, the results are only as good as the content you feed the engine.  Absolutely right.  If you run campaigns designed around static demographics on a behavioral platform you have created a way to “efficiently target crap to your customers”.

Is anybody listening?  If the message is not clear, try this:

Most Marketers are looking to drive “behavior” of some kind – even the Brand folks, who simply have a longer time horizon.  If behavior is the outcome you want, the campaigns must be created around “when”, “what”, and “why”, not “who”.  “When”, “what”, and “why” are behavioral ideas, “who” is a static characteristic (like a demographic) that probably has nothing to do with past or future behavior.

I know, you have probably been told segmenting by demographics is the way to go, or read so somewhere.  Was the source talking about buying media or data-driven marketing?

Sure, if you don’t have any behavior – when buying TV for example – then you go with what you can get.  Some segmentation is always better than none at all.  But if you have behavior, then using demographics to drive campaign segmentation is going to be sub-optimal.

Static characteristics like age and income do not predict behavior.  Behavior is in motion; it changes over time.  You can’t take a static characteristic and expect it to do a very good job predicting behavior because behavior changes over time.  Behavior predicts behavior.

The fact I am a 48 year old male predicts nothing about my behavior.  These characteristics are simply a proxy for buying media against me more efficiently; they really mean nothing when you cross the line into using data sets with actual behavior in them.  The fact I stopped visiting / posting / purchasing or that I am in the top 10% for writing reviews is much more powerful.

When addressing behavioral segments, first ask When?  When did I stop visiting / posting / purchasing?  Over what time period am I in the top 10%?  Am I still in the top 10%?

Then ask, What?  What events led up to my behavior?  What campaign did I come in from, salesperson did I talk to, products did I buy, areas of the site did I visit?  What has happened to me?

Then, understanding my experience, ask Why am I behaving like I am? Then knowing Why (or more likely, making an educated guess), can you think of a message that is going to change my behavior?

Now you are ready to design and execute a campaign that will blow the socks off of anything you can do by knowing I am a 49 year old male, because you can directly address me with a message that is more relevant to me.

Marketers, please take the time to think about “when”, “what”, and “why” in campaign design and execution if using behavioral data, and forget about “who”.  You will be glad you did

Analysts, have you ever run into this problem?  Rich evidence of a behavioral “edge” you might have that is ignored in the creation and execution of the campaign?

P.S.  The glad you did link above shows what you can learn by looking at behavioral segments as opposed to demographics.  All the folks in this test are in the same demographic segment, with a 10% overall response rate to a 20% discount offer – better response than any other demographic segment.  But they sure had different levels of profitability, based on behavior. The more engaged they were – as measured by time since last purchase – the less profitable they were for this campaign.  And you can predict this result, because it will happen every time you use the same behavioral segmentation and offer, with slight variations possible across demographic segments.

Aberdeen on Web Analytics Education

John Lovett at Aberdeen has produced a review of the educational opportunities out there for folks interested in learning web analytics.  It’s a wide ranging piece covering everything from the Yahoo Group to the various agencies to the WAA courses to the Master of Science in Analytics from NC State.  John says:

“Web analytics usage has reached mainstream status with 82% adoption among companies surveyed recently by Aberdeen.  However, a vast range of maturity exists regarding analytics process, data analysis and corporate understanding of web metrics.  A fundamental impediment precluding many companies from building a successful analytics program is a lack of skilled employees required to manage, distribute and analyze web analytics.”

He addresses this situation in two parts:

Vendor sponsored programs and consultants, blogs, and guru sessions

Community forums, industry associations, and academic programs

These are unlocked research reports, no charge to view. 

The NC State effort is quite interesting; they are taking the “blended approach” I feel is where we are headed.  Data is data, behavior is behavior, and many of the offline analytical disciplines have a lot to offer the folks in web analytics.  We’re already seeing web analytics job postings with phrases like “strong knowledge of SAS and SPSS highly desirable” meaning employers are looking for cross-platform, cross-tool, cross-channel analysts.

The folks with this cross-knowledge set who can also “speak business” are going to be a very hot commodity going forward.  Fortunately, most web analysts already “speak business”, it’s part of the WA culture – and speaking business is the hard part for most analytical minds.  Like I said, the data is data, the behavior is behavior – and the tools are just tools.  Web analytics is patient zero, infecting the corporation with a proper analytical culture.

If you’re a web analyst and are offered a chance to do SAS / SPSS / Business Objects / etc. training, I would jump on it.

Thanks John / Aberdeen for a great “Sector Insight” piece of research.