Category Archives: Analytics Education

What Data Mining Can and Can’t Do, Part 2

Analytics Education, DataBase Marketing, Web Analytics

The previous post was about what data mining is good for and what it is not good for, and how to use data mining properly for Marketing efforts.  This post further explains this concept in response to comments received.

Detecting credit fraud, especially with a data set as huge as the one at MCI, is a perfect application for data mining – classification, as in “this is fraud, this is not”.  These are not predictions, they are classifications based on a certain type of behavior that has already occurred.  As long as what a Marketer is really trying to accomplish is classification, then data mining is a great tool.  If you are trying to predict behavior, not so good.

I agree data mining has “real potential is to call attention to things for further investigation” as long as the classification will be actionable, but often times it is not.  There is a great deal of confusion about just what data mining can and cannot do and I’m just trying to bring some clarity to this issue for Marketing folks.

Bottom line: classifying people into “buckets” is not particularly helpful without some end result to act on as a result of having people in these buckets.  Ask yourself: if I know that people differ in a certain way, what will I do with that information, how will I act on it?

The most common mistake in this area is thinking demographics in some way predict behavior.  Demographics are not predictive, they are merely suggestive, yet many marketers cling to demos because that’s what they grew up with.  And then the analysts jump right in and say, “We can segment this population by demographics using data mining!” and you’re right off down the rat hole.  Then the Marketers create programs with an Objective of influencing behavior based on this demographic segmentation and wonder why they don’t work.

I certainly don’t have a problem with using “models” in general to solve Business and Marketing problems – that’s what I do for a living.

What I do have a problem with is the tendency to throw brute force machine learning technology at Marketing problems that ultimately can’t be solved using that particular approach. It’s a waste of time and money.  Paula, I think this is an area similar to your: “If this is the answer, what was the question?”

Said another way, detecting a behavior and predicting one are very different Objectives, and a lot of what you want to do in Marketing is prediction, not detection; it’s a “when” question, not a “who” question.  Often in Marketing, by the time you know “who”, it’s too late to do anything about it.  So Marketers need to know the probability of, the propensity to, not a classification of “who” after something happens.

On the flip side, if I have a prediction or propensity already, and then you want to tell me “who” they are with data mining, that’s fine, provided that information will make any difference.  And here we get to the crux of my comment: knowing who after I have the propensity usually does not make any difference at all.  On this point I am sure there will be a lot of disagreement, but I urge anybody who disagrees to simply test the hypothesis.  Show me the time, money, and effort spent on finding out”who” created enough economic value to pay off the investment, created incremental profit beyond the profit generated by simply understanding the propensity all by itself.

More data is not the answer; only the right data is required.  Huge numbers of models are not the answer either; just because I can segment doesn’t mean that segmentation is worth anything.  Data / model output can be considered as must know, good to know, nice to know, and who cares?  Machine learning technologies seem to drive much more “who cares” than “need to know” output, and people end up drowning in irrelevant noise.   This is not a fault of the technology, but the application of it improperly.

For most Marketing needs, data mining is like “crop dusting with the SST”, to quote a former CEO I worked for.  Discovering a Marketing problem is typically the easy part and doesn’t require data mining; taking the right action to solve the problem is where the difficulty lies and machine learning is not going to provide that answer, despite many people hoping or believing it is true.

Of course, the inability of many Marketers to understand and communicate the actual problem they are trying to solve, and / or the inability of many technology people to turn those requirements into an actionable solution, is a different story that we won’t begin to address in this forum.  To the extent either one is responsible for the misapplication of a certain technology to solving a problem, oh well, where have we heard that before.

I hope I explained my position more clearly this time!

“About the Blog” as a Post

Analytical Culture, Analytics Education, Marketing / Tech Interface

I had a request to publish my “About the Blog” page as a post so people could comment on it.  Here ya go Jacques.

From the Drilling Down newsletter, 12/2004:

What is the number one characteristic shared by companies who are successful in turning customer data into profits?  The company fosters and supports an analytical culture.

Web analytics and Pay-per-Click Marketing in particular have served to teach many people the basics of applying the scientific method to customer data and marketing – creating actionable reporting, tracking source to outcome, KPI’s, iterative testing, etc.  The web has allowed companies to dip a toe into the acting-on-marketing-data waters at relatively low cost and risk when compared with offline projects.  And many have seen incredible ROI.

I think web analytics could be poised in the future to serve a greater role – teaching people / companies the optimal culture for success using analytics, also at relatively low cost and risk.  It’s going to be much harder to drive this concept but more rewarding if as users we can make this happen, because today’s web analysts (and maybe analytical apps) could potentially be among tomorrow’s leaders in a data-based, analytics-driven business world.

For example, do you think analyzing / understanding new interactive data streams where the interface is not a browser will be any different, in terms of the culture required to turn interactive customer data into profitable business actions?  I don’t.

Look, a “request” is a request, whether a click, IP phone call connect, cable TV remote button push, verbal command, card swipe, RFID scan, etc.  You’re still asking a computer to do something.  The request has a source, is part of a sequence (path), and has an outcome. 

Analysis of these requests will face challenges and provide potential benefits similar to those provided right now in web site analytics.  This is the beginning of analyzing the interaction of computers, people, and process.  

Without a doubt, no matter what form these requests take, there will be a “log” of some kind to be analyzed.  Usability?  Conversion?  ROI?  These issues are not going to go away, and companies need to develop a culture that properly embraces analyzing and addressing them.  Companies not developing this culture will find themselves continuing to bump along the “drowning in data” road and will never optimize their interactive customer marketing.

As I see it, here’s the “culture” issue in a nutshell: as a company, you have to want to dig into data and really understand your business.  This pre-supposes that you (as a company) believe that understanding the guts of your business through analytics will drive actions that increase profits.  If the company doesn’t generally support this idea, there is no incentive for anyone to pursue it and the company just happily bumps down the road.

Of course most people don’t really relate to the “company”, but their own division or functional silo.  So you might have manufacturing / engineering groups who live and die through analytics but marketing is not held to the same standards and thought processes.  This is where the idea of Six Sigma Marketing comes in, it’s a “bridge” of sorts that tries to say (perhaps to the CEO and CFO), “Hey folks, if the engineers can engage in continuous improvement through ongoing analytics, so can the Customer Service silo and the Marketing silo and perhaps others.”

At a higher conceptual level, analytical culture takes root when management makes it known they are not afraid of failure, and want employees not to be afraid of it either.  

Another way to say this is experimentation and testing are encouraged throughout the company.  Failure is a regular occurrence, and is even celebrated because through failure, learning takes place.  Show me a company with no failures or that hides failure and I’ll show you a company that is asleep at the switch, afraid of its shadow, a company soon to be irrelevant to the market it serves.

Hand in hand with accepting failure must be continuous improvement.  Even though failure is embraced as a learning tool, the lesson of the failure both prevents it from happening again and results in new ideas with a higher potential for success.  These twin ideas of embracing failure / continuous improvement are at the heart of every business successful in using analytics to improve profitability.

“Evidence” of a company with the right bones to grow an analytical culture is this: you see the various levels of employees working in cross-functional teams with a common problem-solving mission.  Instead of people in a silo groaning about members from other silos being present at a problem-solving meeting, people are instead asking, “Where is finance, where is customer service?”

The most common place “analytics” live in a company is in Finance with the “Financial Analysts”, who are mostly tasked with analysis related to financial controls and producing financial reports.  If marketing or customer service was willing to expose themselves to the rigor of these analysts, they would undoubtedly be able to improve their business areas.  But that exposure takes substantial guts and confidence in your abilities, and a “culture” that supports a scientific process.

And you can’t engage in this process without analytics; success and failure need to be defined and measured.  The easiest way to encourage this culture to take root is to team a department head with a Financial Analyst familiar with the area.  

Often, you find this finance person already has insightful questions that could lead to improvement, but “never asked” because “it’s not my job”.  And often, to make changes in a business today, you need IT support of some kind.  That’s the basic cross-functional unit – Finance rep, IT rep, and a department head.  

I would also argue that if Marketing has a seat at the table in the strategic, “Voice of the Customer” sense (as opposed to being relegated to Advertising, PR, and Creative), then marketing is part of the core unit.  Then you add other disciplines as needed based on the particular problem you are trying to solve.

If the culture is flexible enough, this can turn into “Business SWAT” where the best and brightest cross-functional teams roam through the company as “consultants”, tackling the hardest business problems, which (surprise) are usually cross-functional in nature.  And “blame” is never on the agenda, it’s about “how can we help you make it better?”  You need a culture that is clear about this idea in order for people to expose themselves to the analytics-based scientific process.  Success and failure are defined by the analytics.

If you think about it, web site management ruled by analytics is a microcosm of this Business SWAT set-up.  You have marketing, finance (ROI component), and technology all working together based on the data.  That’s why I think there is a higher mission for the web analytics area / people; they are building the prototype that can teach companies how to go about measuring, managing, and maximizing a data-driven business.

At the highest level of this culture, managers “demand” these SWAT teams because the success rate and business impact is so high.  As the various departments or functional silos produce wins and losses, capital (budget) flows to where the successes are and away from the failures.  When managers see this happening, they jump on board, because they want the budget flowing their way.  This creates a natural economic supply and demand scheme with a reward system for participation built into the process.

One caution: when the culture gets to this level, the analytics group must be sanitized from the reporting hierarchy.  It can’t report to finance, or marketing, or IT anymore.  It has to be completely independent, which usually means reporting directly to the CEO.  There has to be confidence in the integrity of the results of all testing based on standards.  All the little “pools” of analytical work throughout the company must be gathered into one.

What kind of companies do you see really engaging in this kind of culture right now? Those that for legacy reasons have always had access to their operational and customer data and have been using analytics for years.  For these legacy players, web analytics is a “duh” effort – they get it right out of the box, because it’s more of the same to them.  But many types of businesses have not had this access to data before and web analytics is the first taste they are getting of the power and leverage in the scientific method.  I think this “accountability” disease we’ve created in web analytics and search marketing will continue to spread and infect every business unit.

The longer-term question is, can we flip this model over, can the successful culture of cross-functional approach and continuous improvement used in web analytics be used to create a “duh” moment for other areas of the company?  Will “best practices” and success stories create an environment where people say to the (web?) analytics team, “Hey, can I get some of that over here?”  In other words, will the analytical culture develop?

Methinks there is more going on with web analytics than meets the eye; it’s potentially a platform for the creation of a new business culture, a culture based on the scientific method – Six Sigma Everything.  Sure, it’s awkward and maybe the web is not meaningful enough yet to many companies.  But as we thrash all this out, there is something greater being learned here.

Right now, many CRM projects can’t show ROI because nobody knows what to do with the data, how to turn it into action that improves the business.  Sounds very much like web analytics 5 years ago…and look what we talk about now.  KPI’s, turning data into action.  The analytical culture playing out.

What does this mean for the people currently involved in web analytics?  If I was a young web analytics jockey, I would be preparing for the spread of the analytical culture, and seriously thinking about learning some of the tools traditionally used in offline analytics – the query stuff like Crystal Reports, the higher end stuff like SAS, SPSS, and so on.  Search the web for “CHAID” and “CART” and see if you like what you read about these analytical models.  If this kind of stuff interests you, you are much closer to being a business analyst than you think.  And guess what?  Analysts who can both develop the business case and create the metrics and methods for analysis – like you have to do for a web site – are rare.

It takes a particular mind set, and that mind set is not common.  Most of the people with the right mind set go into the hard sciences, but demand on the soft side of business (marketing, customer service, etc.) is just beginning in our data-driven world.  

On the hard side, (with all apologies to the real engineers out there for the exaggeration) the drug works or it doesn’t, the part fits or it doesn’t.  The development of softer-side marketing and service analytical techniques is always going to be populated with a lot more gray area than there is on the hard side, and it takes a special skill to conceive of and develop the metrics required.  But we should be trying to bring the same analytical rigor to the soft side of business that the hard side has always had to deal with.  The trick is to apply that rigor without damaging the mission.

For example, the whole “fire your unprofitable customers” thing from some factions in CRM.  That’s ridiculous.  What you want to do is identify them and then act appropriately, whether that means controlling their behavior, not spending additional resources on them, or not doing the things that create them in the first place.  That’s the gray showing.  You don’t just hit the “reject button” on a customer.

Customer data is customer data.  It’s all going to end up in one place eventually as the analytical culture spreads, and those with the skills to apply the scientific method across every customer data set are going to be rare and in very high demand.  Don’t spend all your spare time watching the Forensic Files on Court TV.  You’re a business analyst.  Get out there and learn the rest of your craft!

And, please consider doing whatever you can, whenever you can, to spread the analytical culture within your company.  If most of what your analytics involve is “online marketing”, reach out to “offline service” or another silo and ask if you can help them with anything.  What’s the call they would like to take less of, can you use the web site to make that happen – and prove that it worked?  Can you use the web site to generate offline ROI?  

Web analysts, you are the cross-functional prototype.  Please teach others how to optimize the entire business.

Measuring Customer Experience ROMI #2: Lab Store – New Customer Kits

Analytics Education, DataBase Marketing, Lab Store, Marketing thru Operations

Here’s another Customer Experience kind of test that proves you can generate incremental profit by improving the Experience.  You just have to make sure customers want the experience “improved”.  This example is from the Lab Store and the ROMI on this little program is a real eye popper.

Back in the old days (meaning the 80’s), what I guess is now called WOW was referred to as “surprise and delight”.  Essentially, this 2-step idea works like this: when you surprise the customer, you really get their attention.  If you can get their attention by surprise and delight them at the same time (instead of pissing them off with your surprise), then you are going to have a more loyal customer.  The trick, of course, is to somehow make more money doing it…

New Customer Kits are a very simple way to do this, and in my remote retailing experience, it works every time.  First impressions, in case you didn’t know, are really important – and especially so in remote retailing, where there is no way for the customer to get any tangible “feeling” for the company.  Sure, you have copy on the web site that paints a picture.  But how many times have people read all this wonderful copy only to be screwed when delivered the tangible experience?

The challenge is to design a kit that is relatively inexpensive yet packs an emotional delight.  Lots of people toss extra stuff for the customer in the first order, but that stuff is usually company-centric, for example, “Here is a magnet with our URL on it” or “Here is a catalog of our other products”.  That’s fine, but it’s neither surprising nor delightful.

Here is what makes up a good New Customer Kit, based on years of testing:

1.  A letter or other message from the company that Welcomes the customer, talks about the people and philosophy behind the company, and reinforces any guarantees or promises that are part of the Brand.  This piece must be written carefully, and from a customer-centric point of view.  No “we we” stuff.

2.  A free gift.  This gift must be related to the merchandise or general category being purchased, and must not be discards, seconds, or defective merch.  Giving a new customer something that is dented or discolored is not a gift, it’s an insult.  Giving a new customer something that is promotional (magnet) may be a gift, but it is expected and not particularly delightful.  Giving a new customer a “gift” because they made a first purchase (Buy today and we’ll include a…) might be delightful but sure is not surprising.  Ignore the above cautions at your own peril.

3.  Free Samples, if relevant to the business.  Anything that is consumable and generates repeat purchase is ideal.

Anyway, I suppose you’re expecting some kind of numbers to go along with all the fuzzy-wuzzy “Oh, if we just make their experience better, they will be more loyal” drivel you hear all the time online.  This is the Marketing Productivity Blog, after all, right?  OK, here are the stats on this technique from the Lab Store.  As usual, this promotion was tested versus control (new customers who did not receive a New Customer Kit are control) and we compare sales activity of both test and control over the next 90 days.  Why 90 days?  Well, if it makes money at 90 days, it sure makes money at 120…

Average cost of New Member Kit (there are several versions) – $.74

Increase in 90-day second purchase rate, test versus control – over 30%

90-day ROMI – 4,891%  ($36.68 in net profit for every $.75 spent)

Surprised and Delighted Customers – Priceless

Now that the bottom line has been presented, the black box folks simply interested in the “what happens” can skip the next part.  If you want to know why it works and maybe learn something useful you can port elsewhere, read on.

New Customer Kits are a great way to shape Theatre of the Mind. 

What you have with a remote retailing customer is a “theatre of the mind” scenario, much like you have in radio advertising.  Customers can’t see or touch you, so “Cues” become extremely important; if you don’t populate the theatre of the mind for the customer, the customer will go ahead and populate it themselves.  If you want some control over the image of your company people create in their head, you need to be proactive.  Theatre of the mind, folks.  Very powerful stuff. 

Our New Customer Kit generates absolutely tons of “Thank You” e-mails from new customers who want to tell us all about how great the experience was purchasing from the Lab Store.  Now, I think you’d agree that purchasing from a web site isn’t a particularly thrilling experience in any way, but if you really listen (and understand a bit of Consumer Psychology) these customers are not really talking about the web site, or even our company.  

What they really are saying is they are very happy with themselves for making a first purchase from us; our actions have confirmed they made a good decision.  Remember, this is remote retailing.  There is risk to the customer, especially on that first purchase; they have no idea if their expectations based on the web site copy are going to match the reality of delivery.  They are concerned about what might happen – will they be proven smart or dumb for taking this risk?

When we deliver the products they ordered in a timely way we meet expectations.  When we deliver these products carefully packed in a pristine new box packed with fresh blank newspaper, we probably exceed expectations by a bit.  But when these new customers get to the Welcome letter, the free gift, and the samples, we blow out their expectations. 

The picture these new customers had in their mind of our company based on the web site experience is then permanently altered; we’re doing brain surgery for 74 cents a head.

Now, I have a question for you – is this program Marketing or Customer Experience Management?