The following is from the August 2007 Drilling Down Newsletter. Got a question about Customer Measurement, Management, Valuation, Retention, Loyalty, Defection? Just ask your question. Also, feel free to leave a comment.
PRIZM Clusters Not as Predictive as Behavior
Q: I am on an interesting project (and my first DB Mktg one): the client has a large loyalty program, and loves his PRIZM clusters. However, when I told him a little more about Recency and suggest that we spread all members across based on it, he was surprised to see that his PRIZM segments were not a predictive indicator at all!
A: Yes, and here is something many people don’t realize about PRIZM and other geo-demo programs, including census-driven. They were developed for site location – where should I put my Burger King, where should I put my mall? They are incredibly useful for this. However, think about all the sample size discussions for web analytics in the Yahoo Web Analytics Group related to A/B testing, and now imagine what your PRIZM cluster looks like.
In most cases, you are talking about 1 or maybe 2 records in a geo location – what is the likelihood these households reflect the overall “label” of the PRIZM cluster? Combine this with the fact that for customer analysis, demographics are generally descriptive or suggestive but not nearly as predictive as behavior and you have a bit of a mess.
Here’s a test for you. It only requires rough knowledge of your neighbors, so should not be very difficult (for most people!)
1. What is your “demographic”?
2. If you were to walk around the block and knock on doors, how many households would you find that are “in your demographic”?
Right. Maybe a handful, unless you live in a brand new housing development or other special situation. Now think about walking your zip code, or walking out 10 blocks or so from your house in any direction, and knocking on doors. Do you find most of these people are in the same demographic as you are? Did you ever find the “cluster average” neighbor?
We certainly know from web analytics that dealing with “averages” can be very dangerous indeed. So too with taking a demographic “average” of a zip or other area and tying it to a specific household. The model falls apart at the household level of granularity.
So now what to you think of all those websites and services that claim to know demographics based on a zip code they captured?
Now, if you think about an e-commerce database, with most records being one of a very few in a zip or cluster, you can see how the cluster demos would really break down at the household level.
Again, nothing wrong with using these geo-demo programs for what they were intended to be used for. When you are looking for a mall location or doing urban planning they can be very helpful. But the match rates at the individual household level are poor.
Couple this with the fact that e-commerce folks are usually looking for behavior from customers, and the fact demographics are not generally predictive of behavior by themselves, and you have yourself analytical stew.
Better than nothing? Absolutely, and for customer acquisition, sometimes all you can get. Best you can be? Not if you have the behavioral records of customers. In fact, what we often see is a skew in the demographics being called “predictive” when the underlying behaviorals are driving action.
In other words, let’s say a series of campaigns generates buyers with a particular demo skew. A high percentage of these Recent responders then respond to the next promotion. If you look just at the demos, you would see a trend and declare the demos are “predictive” of response, even though they are incidental to the underlying Recency behavior.
I suspect something like this was going on with your client. Not looking at behavior, over time the client becomes convinced that the PRIZM clusters are predictive, when for some reason they are simply coincident in a way with the greater power of the behavioral metrics. Given the client has behavioral data, that should be the first line of segmentation.
Q: After reading you for some years, I now understand how one must be very careful with psycho-demographics.
A: Well, at least one person is listening! And now you have seen how this works right before your very own eyes.
I think this situation is really a function of Marketers in general being “brought up” in the world of branding / customer acquisition. Most Marketers come up through the ranks “buying media” or some other marketing activity that focuses on demographics to describe the customer. And most of the college courses and reading material available focus on this function, so even the IT-oriented folks in online marketing end up learning that demographics are really important. And they can be, when you don’t know anything about your target.
Then the world flips upside down on you, and now people are looking at customer marketing, and that’s a whole different ballgame. The desired outcome is “action” that can be measured and the “individual” is the source of that outcome, as opposed to “impressions” and “audience”.
In the past, if your tried and true weapon of choice for targeting was demographics, that is what you reach for as you enter into the customer marketing battle. Problem is, it’s just not the best weapon for that particular marketing engagement.