Jim answers questions from fellow Drillers
(More questions with answers here, Work Overview here, Index of concepts here)
Q: The research folks in my company are trying to convince me that measuring sessions and Page Views per Session is more effective than using Recency and Sessions, as you advocate in your book, for a retention metric.
A: For a content site, the Page Views / Session measure can be used as a measure of visitor quality and appropriate marketing to the right audience – a customer acquisition idea – not retention. And it really needs to be broken out by Source – the average has little actionable meaning. You want to know the Visitor Sources, and then look at this metric by Source. This is still Frequency though – what about visitors who don’t come back?
Q: I am having some difficulty in making a decision regarding this. They want to give me a matrix with Page Views per Session on the Y axis and Total Sessions on the X axis as the “customer retention map”.
A: I’d call that a Customer Activity or Customer Engagement map, I don’t see how it has anything to do with Retention. This is a “Frequency Map” with no Recency, which is more of a visitor quality thing, it doesn’t tell you anything about Retention.
For example, as long as you are growing your visitor base, with this kind of map you would never realize Visitors were defecting until the Visitor totals started to go down. By then, it would be too late to take action, the damage is done and you’re into the death spiral.
Amazon used to boast about how many “best customers” they had based on Frequency. When Wall Street found out 60% of these best customers had not made a purchase in over a year (Recency), the stock was cut in half. Frequency is a measure of Current Value, it doesn’t mean anything for Retention without Recency.
Recent-Frequent visitors are best customers, now and in the future; Frequent visitors who are not Recent are former best customers. Further, without Visitor Source, I don’t know how you take action on the reporting above.
Q: They say that if I need to send messages to only the recent users I could by only querying the users with high sessions that were only generated over the recent past, say, 1 week. This will be enough to target the users who you say are most likely to respond.
A: Well, this is true, they are saying to pull the list by Recency and Frequency. So if targeting an e-mail to most likely to respond is what you want to do, this would work. But it doesn’t give you a “Visitor Retention Map”, if that is what you want.
See a detailed example of such a map here.
The method your folks are suggesting doesn’t tell you how to get and keep Best Visitors, or which Best Visitors seem to be defecting, providing a head’s up on a weak Visitor Source.
So if growing the best Visitor base as rapidly as possible is what you want to do, you need to know if you are churning through a ton of best customers each week. I would be more concerned about mailing to best customers who are not coming back. You can’t do that without Recency.? Sounds to me like they want to take the easy way out. It’s easy to pull a list of people who have been to the site, harder to pull a list of people who have not been there for some period, which is what you want if you intend to act on Retention data.
Q: What are we missing here? Why, if at all, is it more important to gauge Recency and Frequency (sessions) than Sessions and Page Views per Session? It sounds like Page Views per Session is more like “Intensity” than anything else, and I can see how it is also valuable. Can you shed some light here?
A: Yes, Intensity, which implies Quality, as I said earlier – for most sites a desirable thing. It’s just another way to measure Frequency. So if this metric is more important than Total Page Views or Total Sessions to your business, then use it. But without Recency, you have no predictive power. If you are going to do list pulls based on Recency but not track based on Recency, it seems to me you will be missing the data you need to achieve your retention Objective.
How do you know if it’s working?
Under this method, you could start with 10,000 Best visitors and work your way all the way down to 100 that actually still visit, never learning anything or coming up with a plan to reduce defection. Worse, this mass defection could be covered up by Acquisition efforts without the Recency reporting, meaning you’d just be churning Visitors and not making any progress.
This kind of Frequency reporting, at best, tells you what has happened in the past, e.g. at some point you had 10,000 visitors who were considered “Best”. It does not tell you how long it has been since any of these Best Visitors visited the site.
For example, say you are running two ad campaigns. Ad A generates Views/Session of 6.4, Ad B Views/Session of 3.2. Based on the tracking proposed, Ad A generates the “best customers”. So you put all your money into Ad A, and kill Ad B.
Later, you want to do a high-response e-mail, so you pull all the 1 week Recent visitors with high Views per Session, send it out, and get high response. So far so good. On further analysis, you find most of these people originally visited as a result of Ad B. Curious, you look at the Recency of Ad A visitors and find 90% never came back after the 2nd or 3rd visit.
So you have just wasted money and time, because you were not tracking Customer Retention, just using a Retention-oriented metric to pull lists. You’re only hitting people who decide to keep visiting, and as a result, never learn why people stop visiting. Whatever you are doing to create this scenario goes unchecked, and all of a sudden the business is in freefall. The Recency metric can predict this is going to happen for you, give you the head’s up that something is not working for the Visitor segment and allow you to take action.
See the difference? The value of this kind of modeling is to be able to predict the future, not react to the past. I don’t know what you are trying to accomplish in the end, what is the Objective of all this? But assuming high Page Views / Session is good, then you have a way to track Frequency, which is one vector on the Customer Retention Map, the Current Value. But what good is knowing “initial quality” if this quality doesn’t stick over time?
That’s the other half of the Customer Retention map, the Potential Value. And that’s where Recency comes into the picture.
If you intend to increase Retention, try to keep your customers longer, you want to:
1. Understand which Visitor sources create Visitors who don’t come back, and if paying for any of these sources, reallocate the budget to those sources creating Visitors who do come back again and again.
2. Take action with the folks that were high value Visitors who have stopped Visiting. Getting a high response rate from Current Visitors doesn’t imply anything about Retention; it just tells you that engaged visitors are more likely to respond.
What you want to know is what Visitor dis-engagement looks like, how many high value Visitors are you losing? What were their Sources? What kinds of content did they Engage with / Products did they buy? Were they frequent contributors? Did they have Service encounters? Then find out why they are leaving and take action.
Fordetailed examples of Visitor & Customer Retention Mapping, see the Measuring Engagement Series. For examples of how to measure Visitor Recency in Google Analytics, see this post by Avinash.
Jim
Get the book at Booklocker.com
Find Out Specifically What is in the Book
Learn Customer Marketing Concepts and Metrics (site article list)
Download the first 9 chapters of the Drilling Down book: PDF