Category Archives: DataBase Marketing

PRIZM Clusters Not as Predictive as Behavior

DataBase Marketing, Driller Q & A, Newsletters

Jim answers questions from fellow Drillers
(More questions with answers here, Work Overview here, Index of concepts here)

PRIZM Clusters Not as Predictive as Behavior

Q: I am on an interesting project (and my first DB Mktg one): the client has a large loyalty program, and loves his PRIZM clusters. However, when I told him a little more about Recency and suggest that we spread all members across based on it, he was surprised to see that his PRIZM segments were not a predictive indicator at all!

A: Yes, and here is something many people don’t realize about PRIZM and other geo-demo programs, including census-driven. They were developed for site location – where should I put my Burger King, where should I put my mall? They are incredibly useful for this. However, think about all the sample size discussions for web analytics related to A/B testing, and now imagine what your PRIZM cluster looks like.

In most cases, you are talking about 1 or maybe 2 records in a geo location – what is the likelihood these households reflect the overall “label” of the PRIZM cluster? Combine this with the fact that for customer analysis, demographics are generally descriptive or suggestive but not nearly as predictive as behavior and you have a bit of a mess.

Here’s a test for you. It only requires rough knowledge of your neighbors, so should not be very difficult (for most people!)

1. What is your “demographic”?
2. If you were to walk around the block and knock on doors, how many households would you find that are “in your demographic”?

Right. Maybe a handful, unless you live in a brand new housing development or other special situation.  Now think about walking your zip code, or walking out 10 blocks or so from your house in any direction, and knocking on doors. Do you find most of these people are in the same demographic as you are? Did you ever find the “cluster average” neighbor?

We certainly know from web analytics that dealing with “averages” can be very dangerous indeed. So too with taking a demographic “average” of a zip or other area and tying it to a specific household. The model falls apart at the household level of granularity.

So now what to you think of all those websites and services that claim to know demographics based on a zip code they captured?

Now, if you think about an e-commerce database, with most records being one of a very few in a zip or cluster, you can see how the cluster demos would really break down at the household level.

Again, nothing wrong with using these geo-demo programs for what they were intended to be used for. When you are looking for a mall location or doing urban planning they can be very helpful. But the match rates at the individual household level are poor.

Couple this with the fact that e-commerce folks are usually looking for behavior from customers, and the fact demographics are not generally predictive of behavior by themselves, and you have yourself analytical stew.

Better than nothing?  Absolutely, and for customer acquisition, sometimes all you can get. Best you can be? Not if you have the behavioral records of customers. In fact, what we often see is a skew in the demographics being called “predictive” when the underlying behaviorals are driving action.

In other words, let’s say a series of campaigns generates buyers with a particular demo skew. A high percentage of these Recent responders then respond to the next promotion. If you look just at the demos, you would see a trend and declare the demos are “predictive” of response, even though they are incidental to the underlying Recency behavior.

I suspect something like this was going on with your client. Not looking at behavior, over time the client becomes convinced that the PRIZM clusters are predictive, when for some reason they are simply coincident in a way with the greater power of the behavioral metrics. Given the client has behavioral data, that should be the first line of segmentation.

Q: After reading you for some years, I now understand how one must be very careful with psycho-demographics.

A: Well, at least one person is listening!  And now you have seen how this works right before your very own eyes.

I think this situation is really a function of Marketers in general being “brought up” in the world of branding / customer acquisition. Most Marketers come up through the ranks “buying media” or some other marketing activity that focuses on demographics to describe the customer. And most of the college courses and reading material available focus on this function, so even the IT-oriented folks in online marketing end up learning that demographics are really important. And they can be, when you don’t know anything about your target.

Then the world flips upside down on you, and now people are looking at customer marketing, and that’s a whole different ballgame. The desired outcome is “action” that can be measured and the “individual” is the source of that outcome, as opposed to “impressions” and “audience”.

In the past, if your tried and true weapon of choice for targeting was demographics, that is what you reach for as you enter into the customer marketing battle.Problem is, it’s just not the best weapon for that particular marketing engagement.

Jim

Get the book at Booklocker.com

Find Out Specifically What is in the Book

Learn Customer Marketing Concepts and Metrics (site article list)

Download the first 9 chapters of the Drilling Down book: PDF 

Will Work for Data

Analytical Culture, Analytics Education, DataBase Marketing

But will do a sub-optimal job…

Trying to catch up on what is going on in the analytics blogosphere, and it seems like I’m seeing a common thread – we’re getting much better at analyzing customer data, but whoever is in charge of Turning Customer Data into Profits is not quite with the program yet. 

Based on my experience, and assuming the people responsible are Marketing folks, the challenge to solving this problem often lies in understanding the difference between executing against behavioral data and executing against data about “characteristics” like demographics.

Marketing is not always about buying mass media, yet most Marketing people have never had to create and execute a campaign using behavioral data against a behavioral Objective.  So they do what they have always done – they create campaigns based on characteristics – and then execute against behavioral objectives using behavioral data.

This is a recipe for sub-optimal performance.  It’s like buying a car with a high performance engine then putting the cheapest gas in it you can find and never getting a tune up.  Sure, the car will run, but it’s not going to run very well, and you sure are not going to win any races with the competition.  Provided, of course, they don’t treat their car the same way.

For example, Ron is commenting on weak segmentation practices and lack of understanding the new customer experience in banking.  He is absolutely right.  Segmenting by “number of products” is often a static characteristic; segmenting by “change in number of products” is behavioral and many times more profitable.  As for new customer experience, the initial experience defines a customer’s “view” of the company and I don’t think I have to explain the importance of that.

Kevin is bemoaning the lack of temporal segmentation and use of appropriate creative for this segmentation by many e-mail folks.  He is absolutely right.  You want to speak to the customer based on their level of engagement with the company, not in terms of static perceptions.

Avinash perceives a problem coming down the road with behavioral targeting, that is, while the machine is smart, the results are only as good as the content you feed the engine.  Absolutely right.  If you run campaigns designed around static demographics on a behavioral platform you have created a way to “efficiently target crap to your customers”.

Is anybody listening?  If the message is not clear, try this:

Most Marketers are looking to drive “behavior” of some kind – even the Brand folks, who simply have a longer time horizon.  If behavior is the outcome you want, the campaigns must be created around “when”, “what”, and “why”, not “who”.  “When”, “what”, and “why” are behavioral ideas, “who” is a static characteristic (like a demographic) that probably has nothing to do with past or future behavior.

I know, you have probably been told segmenting by demographics is the way to go, or read so somewhere.  Was the source talking about buying media or data-driven marketing?

Sure, if you don’t have any behavior – when buying TV for example – then you go with what you can get.  Some segmentation is always better than none at all.  But if you have behavior, then using demographics to drive campaign segmentation is going to be sub-optimal.

Static characteristics like age and income do not predict behavior.  Behavior is in motion; it changes over time.  You can’t take a static characteristic and expect it to do a very good job predicting behavior because behavior changes over time.  Behavior predicts behavior.

The fact I am a 48 year old male predicts nothing about my behavior.  These characteristics are simply a proxy for buying media against me more efficiently; they really mean nothing when you cross the line into using data sets with actual behavior in them.  The fact I stopped visiting / posting / purchasing or that I am in the top 10% for writing reviews is much more powerful.

When addressing behavioral segments, first ask When?  When did I stop visiting / posting / purchasing?  Over what time period am I in the top 10%?  Am I still in the top 10%?

Then ask, What?  What events led up to my behavior?  What campaign did I come in from, salesperson did I talk to, products did I buy, areas of the site did I visit?  What has happened to me?

Then, understanding my experience, ask Why am I behaving like I am? Then knowing Why (or more likely, making an educated guess), can you think of a message that is going to change my behavior?

Now you are ready to design and execute a campaign that will blow the socks off of anything you can do by knowing I am a 49 year old male, because you can directly address me with a message that is more relevant to me.

Marketers, please take the time to think about “when”, “what”, and “why” in campaign design and execution if using behavioral data, and forget about “who”.  You will be glad you did. 

Analysts, have you ever run into this problem?  Rich evidence of a behavioral “edge” you might have that is ignored in the creation and execution of the campaign?

P.S.  The glad you did link above shows what you can learn by looking at behavioral segments as opposed to demographics.  All the folks in this test are in the same demographic segment, with a 10% overall response rate to a 20% discount offer – better response than any other demographic segment.  But they sure had different levels of profitability, based on behavior. The more engaged they were – as measured by time since last purchase – the less profitable they were for this campaign.  And you can predict this result, because it will happen every time you use the same behavioral segmentation and offer, with slight variations possible across demographic segments.

What Data Mining Can and Can’t Do, Part 2

Analytics Education, DataBase Marketing, Web Analytics

The previous post was about what data mining is good for and what it is not good for, and how to use data mining properly for Marketing efforts.  This post further explains this concept in response to comments received.

Detecting credit fraud, especially with a data set as huge as the one at MCI, is a perfect application for data mining – classification, as in “this is fraud, this is not”.  These are not predictions, they are classifications based on a certain type of behavior that has already occurred.  As long as what a Marketer is really trying to accomplish is classification, then data mining is a great tool.  If you are trying to predict behavior, not so good.

I agree data mining has “real potential is to call attention to things for further investigation” as long as the classification will be actionable, but often times it is not.  There is a great deal of confusion about just what data mining can and cannot do and I’m just trying to bring some clarity to this issue for Marketing folks.

Bottom line: classifying people into “buckets” is not particularly helpful without some end result to act on as a result of having people in these buckets.  Ask yourself: if I know that people differ in a certain way, what will I do with that information, how will I act on it?

The most common mistake in this area is thinking demographics in some way predict behavior.  Demographics are not predictive, they are merely suggestive, yet many marketers cling to demos because that’s what they grew up with.  And then the analysts jump right in and say, “We can segment this population by demographics using data mining!” and you’re right off down the rat hole.  Then the Marketers create programs with an Objective of influencing behavior based on this demographic segmentation and wonder why they don’t work.

I certainly don’t have a problem with using “models” in general to solve Business and Marketing problems – that’s what I do for a living.

What I do have a problem with is the tendency to throw brute force machine learning technology at Marketing problems that ultimately can’t be solved using that particular approach. It’s a waste of time and money.  Paula, I think this is an area similar to your: “If this is the answer, what was the question?”

Said another way, detecting a behavior and predicting one are very different Objectives, and a lot of what you want to do in Marketing is prediction, not detection; it’s a “when” question, not a “who” question.  Often in Marketing, by the time you know “who”, it’s too late to do anything about it.  So Marketers need to know the probability of, the propensity to, not a classification of “who” after something happens.

On the flip side, if I have a prediction or propensity already, and then you want to tell me “who” they are with data mining, that’s fine, provided that information will make any difference.  And here we get to the crux of my comment: knowing who after I have the propensity usually does not make any difference at all.  On this point I am sure there will be a lot of disagreement, but I urge anybody who disagrees to simply test the hypothesis.  Show me the time, money, and effort spent on finding out”who” created enough economic value to pay off the investment, created incremental profit beyond the profit generated by simply understanding the propensity all by itself.

More data is not the answer; only the right data is required.  Huge numbers of models are not the answer either; just because I can segment doesn’t mean that segmentation is worth anything.  Data / model output can be considered as must know, good to know, nice to know, and who cares?  Machine learning technologies seem to drive much more “who cares” than “need to know” output, and people end up drowning in irrelevant noise.   This is not a fault of the technology, but the application of it improperly.

For most Marketing needs, data mining is like “crop dusting with the SST”, to quote a former CEO I worked for.  Discovering a Marketing problem is typically the easy part and doesn’t require data mining; taking the right action to solve the problem is where the difficulty lies and machine learning is not going to provide that answer, despite many people hoping or believing it is true.

Of course, the inability of many Marketers to understand and communicate the actual problem they are trying to solve, and / or the inability of many technology people to turn those requirements into an actionable solution, is a different story that we won’t begin to address in this forum.  To the extent either one is responsible for the misapplication of a certain technology to solving a problem, oh well, where have we heard that before.

I hope I explained my position more clearly this time!