Category Archives: DataBase Marketing

What Data Mining Can and Can’t Do, Part 2

Analytics Education, DataBase Marketing, Web Analytics

The previous post was about what data mining is good for and what it is not good for, and how to use data mining properly for Marketing efforts.  This post further explains this concept in response to comments received.

Detecting credit fraud, especially with a data set as huge as the one at MCI, is a perfect application for data mining – classification, as in “this is fraud, this is not”.  These are not predictions, they are classifications based on a certain type of behavior that has already occurred.  As long as what a Marketer is really trying to accomplish is classification, then data mining is a great tool.  If you are trying to predict behavior, not so good.

I agree data mining has “real potential is to call attention to things for further investigation” as long as the classification will be actionable, but often times it is not.  There is a great deal of confusion about just what data mining can and cannot do and I’m just trying to bring some clarity to this issue for Marketing folks.

Bottom line: classifying people into “buckets” is not particularly helpful without some end result to act on as a result of having people in these buckets.  Ask yourself: if I know that people differ in a certain way, what will I do with that information, how will I act on it?

The most common mistake in this area is thinking demographics in some way predict behavior.  Demographics are not predictive, they are merely suggestive, yet many marketers cling to demos because that’s what they grew up with.  And then the analysts jump right in and say, “We can segment this population by demographics using data mining!” and you’re right off down the rat hole.  Then the Marketers create programs with an Objective of influencing behavior based on this demographic segmentation and wonder why they don’t work.

I certainly don’t have a problem with using “models” in general to solve Business and Marketing problems – that’s what I do for a living.

What I do have a problem with is the tendency to throw brute force machine learning technology at Marketing problems that ultimately can’t be solved using that particular approach. It’s a waste of time and money.  Paula, I think this is an area similar to your: “If this is the answer, what was the question?”

Said another way, detecting a behavior and predicting one are very different Objectives, and a lot of what you want to do in Marketing is prediction, not detection; it’s a “when” question, not a “who” question.  Often in Marketing, by the time you know “who”, it’s too late to do anything about it.  So Marketers need to know the probability of, the propensity to, not a classification of “who” after something happens.

On the flip side, if I have a prediction or propensity already, and then you want to tell me “who” they are with data mining, that’s fine, provided that information will make any difference.  And here we get to the crux of my comment: knowing who after I have the propensity usually does not make any difference at all.  On this point I am sure there will be a lot of disagreement, but I urge anybody who disagrees to simply test the hypothesis.  Show me the time, money, and effort spent on finding out”who” created enough economic value to pay off the investment, created incremental profit beyond the profit generated by simply understanding the propensity all by itself.

More data is not the answer; only the right data is required.  Huge numbers of models are not the answer either; just because I can segment doesn’t mean that segmentation is worth anything.  Data / model output can be considered as must know, good to know, nice to know, and who cares?  Machine learning technologies seem to drive much more “who cares” than “need to know” output, and people end up drowning in irrelevant noise.   This is not a fault of the technology, but the application of it improperly.

For most Marketing needs, data mining is like “crop dusting with the SST”, to quote a former CEO I worked for.  Discovering a Marketing problem is typically the easy part and doesn’t require data mining; taking the right action to solve the problem is where the difficulty lies and machine learning is not going to provide that answer, despite many people hoping or believing it is true.

Of course, the inability of many Marketers to understand and communicate the actual problem they are trying to solve, and / or the inability of many technology people to turn those requirements into an actionable solution, is a different story that we won’t begin to address in this forum.  To the extent either one is responsible for the misapplication of a certain technology to solving a problem, oh well, where have we heard that before.

I hope I explained my position more clearly this time!

More on Customer LifeCycle / LifeTime Value

Analytical Culture, DataBase Marketing

Adelino has chimed in with some great thoughts on LifeTime Value and ways to execute against it.  I’m all for a practical approach, which he has outlined.

A couple of further ideas:

1.  As counter-intuitive as it might seem, I think we’re better off talking about LTV as a concept and not a “number” in most industries – especially when folks are just beginning to look at this area. 

The idea that customers have “future value” is not a hard one to get across, the idea you can triangulate a specific “number” representing this value is difficult. Tactically (which is what most folks care about), I like to use the idea of “relative LTV” or Potential Value – you don’t need to know a specific number, you just need to know which customers have higher versus lower Potential Value and allocate attention and budget appropriately. This approach allows people to execute against Adelino’s final four points without the mind-numbing LTV math.

Another way to say this: if you manage Potential Value at discrete points in time – say as monthly “snapshots” – the “movie” of LifeTime Value will take care of itself.  For the more aggressive math folks, this begins do look like a “value curve” for each customer or segment where you can look at the slope of the curve to predict Potential Value.  Go all the way to derivatives if you want to!

2.  There is still a need to bolt all this directly into Finance and the Periodic accounting system.  If Marketers or IT folks can do that, now you have something really cooking.  The best way I have found to sit down and start talking about this is to break out Customer Productivity by Financial period, as explained here.

This approach gets Finance to start wrapping their heads around “Potential Value” as opposed to Current Value.  You could use Adelino’s “Profitable, Marginal, Unprofitable” model in much the same way – just configure the customer chart so that it explains why the past Financial Quarter was Profitable, Marginal, or Unprofitable.  “We had a great quarter because the % of Profitable customers rose while the % of Unprofitable customers fell”. 

They need to see it right in front of their Financial eyes.  You get a couple of Quarters in a row where your analysis lines up with the Quarter, and then you start predicting the Financial outcome of Quarters based on the Customer Value Model.  Once you lock the customer analysis to the accounting analysis like this, you’re in, because the next question from Finance is:

Can you do anything to affect this model?

Tracking UnTrackable Campaigns

Analytical Culture, DataBase Marketing, Driller Q & A, Newsletters

Jim answers questions from fellow Drillers
(More questions with answers here, Work Overview here, Index of concepts here)

(Any Tracking is Better than No Tracking at All)

Q: I have a background in direct marketing and the measurement of campaigns using defined sources.  Now I am working at a technical or 2 year college and I’m trying to use my previous knowledge and experience to measure our return on marketing efforts in recruiting new students and converting them from prospects to enrolled students at the school.

I’m looking to measure events and all of the advertising and promotion used to communicate those events as well as other campaigns. It’s difficult to measure since it may include newspaper ads, direct mail, posters, etc. – not just direct mail with a measurable list to refer to for tracking response.

Plus, trying to get the admissions folks to track leads can be difficult. They also want to track return on publications (brochures, flyers, etc.)….not sure how one would do that if they are not mailed to a given measurable list?

I’m looking to track ROI based on the whole equation….from the # of inquiries who came to an event or responded to whatever all the way to whether they matriculated and enrolled in courses.

A: I can feel your pain!

I don’t think there are any easy answers to this. You could simply measure what you can measure through the traditional direct methods you are familiar with, and let the rest “ride”. Or, you can try to selectively determine, as best you can, what the value of all these other activities is by engaging in some kind of testing. This will take some institutional willpower and is probably something you can’t do on your own. In other words, I encourage you to start evangelizing the next generation of marketing measurement at the school.

From your title, I perceive you don’t “report to” marketing, but some higher institutional level responsible for Quality and perhaps Productivity / Accountability for funds that are spent? It sounds to me like your unit might perhaps report into the Financial area of the school at a higher level, and if so, that’s good!

I think I would simply start the conversation with folks in the financial area about some of these issues, and see if you can create some simple tests to get some “direction” on the contribution of the various marketing outlets.

For example, every publication should contain some kind of tracking device. Sometimes you have to be creative with this idea and it won’t always be accurate, but it’s better than nothing at all. If response is generally by phone, then try to get a unique phone number for newspaper ads, brochures, etc. If response is driven to the web site, get unique URL’s put on each document. If response is filling out a sheet or card at an event, have them numbered or coded in some way. Then of course, you need to get the response information – number of phone calls to each number, number of visits to each unique web site URL, number of response cards mailed or turned in.

Implementing a program like this, then finding and getting access to the response info may not be easy, and that’s why it would help to have a “higher power”, particularly a financial one, backing this effort. It’s pretty amazing what people will do when, for example, the people who control the budget for an area say, “You will participate in this tracking program”.

Alternatively, you could go with a test / control kind of scenario where during a quarter, you leave out one particular marketing effort and see if there is an impact on overall Marketing Productivity. This is more of a “marketing mix” kind of approach and not without some problems, including proving the missing marketing effort was responsible or not for changes in Productivity. You have to think about how you might pin these issues down in advance – for example, do you have good baselines for “normal” activity?

Whether or not you decide to pursue all of this is somewhat of a personal choice. Some analysts simply don’t think it is their “job” to help create measurable structures – they only measure what can be measured. Others see the difficulty as a challenge, and want to help build out the structure. Clearly, if you are going to eventually be responsible for measurement, being a part of the team constructing the measurement paths is a real advantage to you. It will involve some politics, but analytics always involves politics at some level. I encourage you to seek out the support you need to make this work.

If there is “pressure” for measurement, someone wants it to happen. Start by finding these people and having a conversation about how it could happen, the strengths and weaknesses of the measurements, the internal challenges you will face.

When taking on something like this, it’s usually best not to try to change the world all at once, but one step at a time. So, for example, looking at the overall “unmeasurable budget”, what is the largest line item? If it’s “newspaper”, that’s a place you probably have the largest leverage. Implement there first, keeping in mind that this single implementation might help you down the road. For example, getting a unique phone number and results tracking for newspaper may teach you a lot about how to get this done for other marketing devices.

The finance people should be able to provide you with some idea of the net “margin” of a course and any other financial ideas that come into play. Then it’s a matter of asking if the spend on the media generated positive results. If the margin on a course is $500, a newspaper ad costing $1000 that only generates 1 student is not a great investment – but it might be the best one relative to other vehicles. This part is not really your call. Your job is to bring the data to life so that people can understand what they are spending and what they are getting.

There could be plenty of reasons why “losing $500” on a newspaper ad is OK – there is “brand exposure”, for example. In this case, the brand exposure only costs $500 versus a perception that it costs $1000, if the student generated is included in the formula. This may be a very positive result of the measurement for many folks in the institution. That judgment is really for someone else to make. Now at least they are making it on a full set of facts as opposed to perceptions they have about cost.

Q: However, how long do you keep measuring enrollments…they may not enroll based on one campaign…might take a few hits before they actually become students.

A: Sure. What seems reasonable? Given an annual budget cycle, let’s say reasonable is 12 months. One benefit from your tracking is you will be able to probably put some numbers against this eventually. If you get calls to a brochure number 2 years after it was issued, then the number is 2 years for a brochure. Newspaper calls stop coming in at 4 weeks, it’s 4 weeks for newspaper.

Q: The other issue: the marketing folks only want to measure up to inquiries–what they have control over.  What’s the best way to only measure the return on that…it’s before a “sale” or “enrollment” even occurs, so the “profit” is not booked.

A: Well, sometimes you simply have to decide what is “best available”. You certainly can start by measuring inquiries, especially since it’s pretty clear in this case marketing lacks some control over key conversion elements – financial aid, student abilities, and so forth. Down the road, it’s possible that certain types of media generate lower quality inquiries with lower conversion rates. You will get there over time.

For now, you could apply the “average conversion” to any lead to get down to the financial part of the game. If all leads on average convert at 25%, then just use that.  Then when tracking gets a wider reach, try to drill into it more deeply. To do this, you’d have to get access to enrollment data, of course. But you don’t have to get “all the data, all the time”. You could do a sample of a couple of months and go through it by hand to match back to inquiries, if you have to. You certainly would not be the first to do something like this to pin down an issue – it happens all the time.

Whether you want to do something like a “by hand count” or see it as part of your job is really more of a personal choice.  You can certainly – and analysts often do – blame a lack of knowledge on system problems, politics, whatever.  Just can’t get the data.  For some people “don’t know” is not acceptable – they have to find the answer, whatever way they have to do it – even if it is by hand!

Q: Does your book give an education example? We’re not “selling” a product and “sales” deals with # of credits taken by students & price per credit plus funding we receive from the state gov’t based on the number of FTEs generated. It seems like such a different animal so I’m struggling to figure how to do ROI for an educational services provider. I have created an Excel template based on how I calculated ROI for other industries but haven’t tried it yet.

A: Sounds like the “profit” side of it is a bit complex with the outside funding, but I’m sure you can get to a “value per FTE” somehow. Just start with something, and make it better as you move along. If you have to, simply take “all revenue” divided by number of FTE’s and you at least have a place to start.

The book doesn’t have extensive educational examples but here are some related topics from the newsletter:

Predicting Student Churn

Profiling Library Customers

I have a lot of interest in these educational measurement scenarios for a variety of reasons.  Keep me posted on how you are getting along and ask any questions you might have as you make your way through?  If I can be of help let me know!

Jim

Get the book at Booklocker.com

Find Out Specifically What is in the Book

Learn Customer Marketing Concepts and Metrics (site article list)

Download the first 9 chapters of the Drilling Down book: PDF