Monthly Archives: December 2007

Marketing into a Downturn

The following is from the December 2007 Drilling Down Newsletter.  Got a question about Customer Measurement, Management, Valuation, Retention, Loyalty, Defection?  Just ask your question.  Also, feel free to leave a comment. 

Want to see the answers to previous questions?  The pre-blog newsletter archives are here.

Q:  I have been asked to create a whitepaper on marketing strategy and tactics for a down or recessionary market.  In your studies and travels have you come across any literature or have thoughts of your own that I may quote?

A:  Well, I suppose someone has written something about it somewhere.  The trades write about it for every downturn! But I don’t know of any primary work on the topic – case studies, research, etc.

I do know that when we get into a down / recessionary market my phone rings more and I work a lot harder.  The “new client” customer retention business is counter-cyclical; people always wake up during the soft times and say,  “Hey, if we can’t drive new customer volume, maybe we can sell more to existing customers!”.  You know, the CEO or somebody read that somewhere…

The problem with this kind of thinking is, in most cases, it’s already too late to do anything about customer retention.  That’s not something people generally want to hear.  I then say, “The economy is cyclical.  Do you want to be prepared for the next downturn?”

The people who answer yes to that question will often become clients; those looking for the “quick fix” generally won’t become clients – but they call again into the next downturn…

It’s a strategy thing, you know?  Long term thinking?  But I digress…

The insidious thing about customer defection is that it’s always there, eroding the asset base, wasting away the hard work.  But people don’t see it until the flow of new customers shrinks, and then all of a sudden, the defection issue is laid bare. 

This is why the retention business is so counter-cyclical; why “discovery” comes in  the downturns.

What you normally find is whatever business change / policy / product is causing customer defection, it takes as long to build up the customer asset again as it did to destroy it. Here is a real-world example.

A retailer makes a significant change in the types of products it sells, because it wants to “attract more new customers”.  For existing customers, revenue per customer starts to fall.  This fact is masked on the revenue side by the attraction of new customers to the new products – for a while.  But it ends up these new customers, in terms of revenue per customer, have a value about 30% less than the old customers.  So even though new customer adds remain consistent, sales start to drop, and over time drop by 30% as old customers defect and are replaced by the new customers worth 30% less.

Two years into this process, a downturn in the economy causes more attention and analysis of the customer base, and this issue is exposed.  Surprise!  The newer kind of customers defect at a higher rate and in a shorter time than the old type of customers.

New management is brought in, and they decide to go back to selling more of the “older” product to attract the higher value customer.  Once they make the switch, it takes just as long for sales to get back to where they were as it did to create this problem in the first place – 2 (very long) years.

And that’s why it is so tough to deliver a “quick fix” to these kinds of problems.  They are systemic in nature and because you are talking about the value of a customer over time, take time to fix.

So, it may well be that your advice should ultimately be “use this downturn to prepare for the next one”, if you know what I mean.  Investigate, learn, and understand what happens this time, so you know what to do next time.  In terms of action items, a few:

1. Analyze the customer base, to understand the source of customer value.  Who are the best customers, where do they come from? Which media, sales persons, product lines, services, geographies, etc. create the “best  customers” for the business?

2. Analyze these best customers, and understand their behavior.  What would be a warning sign that these best customers – who are probably responsible for the lion’s share of your profits – are cracking into the downturn?  Slowdown in orders per month, average order size, number of contracts, whatever the relevant metrics are.

3. Track a handful of these customer metrics and see how they change as the economy slows.  These metrics will be a map for predicting actual trouble the next time – predicting trouble even before everyone is already talking about “a downturn”.  This gives you the extraordinary advantage of lead time over your competition in reacting to the downturn in business.

4. Complete the same 3 steps above for medium value customers and low value customers, if you have the resources.

5. Now, fully understanding what you have to work with (perhaps for the 1st time?), what is the strategy for a downturn?  Generally, it would consist of a reallocation of resources away from lower productivity to higher productivity activity, in order of importance:

a. For best customers, how do we keep them? 
b. For mid value customers, how do we grow them?
c. For low value customers, how do we reduce costs to acquire or service them?  Note I do not advocate “firing” customers, but you certainly can cut back on acquiring as many low value ones.

For each group, you should have a specific (and probably different) strategy and set of tactics.  What a lot of folks don’t understand is there is almost always a truly remarkable difference between these customer groups, and any “one size fits all” edict or direction is bound to screw up the business,  just like the example of the “new customer” effort from the retailer above.

For example, we know that marketing spend generally softens in a downturn.  Companies cut back on marketing because they feel like they are “pushing on a string”.  They cancel or don’t buy advertising, they fire salespeople.  This is the wrong move.  The old saw about buying more marketing into a downturn to “grab share” can also be the wrong move, though has some “accidental” positive effects.

The company should invest in more marketing, but not across the board.  They should buy the right marketing, the marketing that generates the best quality customers.

They should reallocate marketing resources away from generating “c” customers towards generating “a” customers.  If you know trade shows generate leads which turn into “a ” customers and online ads generate leads that turn into “c” customers, you take the money you spend online and book more trade shows.  You let go of salespeople that generate “c” customers and use that salary to bonus salespeople generating “a” customers.

Of course, this analysis and planning is an exercise that should be done all the time, not just into a downturn.  A business should always be trying to understand where customer value comes from and how it is created.  But unfortunately, this issue most often comes up going into a downturn.

You’ll have to excuse me now, the phone is ringing again…

Comments or questions?  Does your company have a “downturn plan”?  Or is it business as usual, just less marketing activity across all the channels?

Share:  twittergoogle_plusredditlinkedintumblrmail

Follow:  twitterlinkedinrss

Control Group Series

This post is an index for the Control Group series.  The following posts were written sequentially but appear on the blog in reverse chronological order which makes a hell of a mess of trying to understand a somewhat complicated topic.  So instead, try reading them sequentially using this index:

Why Use Control Groups?

Control Group Benefits

Culture of Control (Groups)

Are You in Control?

Poison Control

Share:  twittergoogle_plusredditlinkedintumblrmail

Follow:  twitterlinkedinrss

Poison Control

This post is part of a series on control groups.  The first post is here, a list of all posts in the series here

Using control groups standardizes success tracking across:


so that you begin to really understand what types of marketing create the most value.  There’s only a couple of things left you need to know to start using this gold standard of customer campaign measurement.

I would be remiss if I didn’t at least warn you once to make sure you use a true random sample of the campaign population for the control group.  The direct marketing road is littered with the bodies of those who failed to create a truly random control group for one reason or another, usually accidently, sometimes intentionally. 

For example, they sort by customer number lowest to highest then truncate sample selection before the whole population has been sampled, not realizing the lower the customer number, the longer the person has been a customer.  This creates a bias in control towards “older” customers and screws up the result.  Another common mistake is while trying to make sure the sample is random from a demographic perspective, they end up with a behavioral bias like a higher percentage of Recent buyers in Control than in Test.  There’s nothing that will make your campaign look like it sucked more than stacking Control with customers more likely to respond than those in Test!

The final issue I’d like to bring up is the “organizational stamina” required to execute a controlled testing program. 

In large organizations, a challenge you may encounter is having other people’s campaigns “poison” your control or test groups.  The whole idea of the control is to have this group different in only one way from the test group – they don’t receive your campaign. 

What can happen is someone working with a different segmentation scheme can end up targeting portions of your test or control group, and now you don’t have a controlled test anymore – the control or test has been “poisoned”.

Just to be clear, if the test and control groups are targeted equally, then your test should still be valid, though the overall outcome might be different.  For example, let’s say you have your test and control groups and the company decides to drop a newsletter or announcement to all customers.  Since both test and control will be exposed equally to this newsletter, the incremental effects of your campaign should be preserved. 

Likewise if a national TV campaign is launched.  Your campaign might perform better overall because of the TV, but the lift you get in test versus control should be the same because the TV should affect both test and control equally.

In large organizations where many different groups access the same customer or prospect database, you can see how this poisoning of controlled tests would get to be a mess in quick order.  Without coordination, people would be stomping all over the tests by targeting a piece of a control here and a piece of test there. 

In orgs that are serious about Marketing Productivity, you do typically see a gatekeeper of some kind at the database, making sure that new list pulls do not interfere with any controlled tests that are running.  And yes, sometimes you have to wait to execute your test because there simply are not enough names to go around for the segment you want.  But this is a small price to pay compared to the total chaos of not ever knowing which marketing really works and which does not.

Clearly, there are some Marketing folks who don’t care to know how a campaign really works; “response” is just fine.  In fact, marketing chaos in the database is good for these folks.  Chaos is a fantastic barrier to accountability and the Marketers can just claim ignorance of this control group issue.  That is, until someone with a background in Business Intelligence asks why controls are not being used – and that will not be a pretty day for the Marketer.

But for the analysts out there, I really think it is your duty to start looking at the use of control groups.  Try it a few times and see what you get.  I guarantee you’ll be surprised, and the data you see will open the door to new kinds of thinking and more effective marketing programs for your customer base.

Share:  twittergoogle_plusredditlinkedintumblrmail

Follow:  twitterlinkedinrss

Are You in Control?

This post is part of a series on control groups.  The first post is here, a list of all posts in the series here

Mike Moran recently wrote about how Search Marketing is Direct Marketing.  I myself commented “the Web is a direct marketing machine” back in 2001 when most people hated the idea of PPC marketing and thought it would never catch on.

Most of the critical breakthroughs in optimizing online marketing have been based on direct or database marketing principles that have been around for decades.  In my last post on Control Groups, I said “the insights you will get from using controls will be mind blowing.  You will begin to really understand customer behavior, and that’s the first step to creating truly game-changing customer marketing campaigns”.

I have some examples for you.

Check out this list detailing some of those insights.  Sure, they are in the form of “mistakes” but they are insights nonetheless.  See 41 Timeless Ways to Screw Up Direct Marketing by Nicholas J. Radcliffe.

The interesting thing about this list is most of these mistakes can only be identified if you are using control groups; that’s how important the concept is to customer-centric marketing.    For some mistakes on this list, you will think to yourself, “How could they ever measure that?”

The answer is one you are familiar with: repeated testing, in this case over many different industries and using many different data sets.  But you have to add controls to the test or you won’t see the effects.

Many of these mistakes are things you hear the CRM / customer-centric / CGM pundits talk about all the time, stuff like talking down to the customer, over-communicating, or being intrusive.  But these same folks never offer any conclusive proof of the financial damage these acts can cause; it’s all “gut feel”. 

How would you like to be able to prove what the damage caused by reckless marketing is really worth?

Online marketers are currently making many of these same 41 mistakes – they just don’t know it yet.  #17 and #19 are going to be very disruptive when they become widely understood.  If you want to understand more about these mistakes, a specific example is here or for a broader framework to work from, see here.

But the real question at hand is this: Will you be a driver of the next level of achievement in online customer marketing by suggesting (and eventually requiring) the use of Control Groups?

In the final post of this series, we’ll touch on two challenges with the implementation of control groups.

Share:  twittergoogle_plusredditlinkedintumblrmail

Follow:  twitterlinkedinrss

Culture of Control (Groups)

This post is part of a series on control groups.  The first post is here, a list of all posts in the series here

There are a couple of analytical culture issues I’d like to touch on with using control groups.  Control groups are the gold standard in customer marketing campaign measurement, and at some point, you will be asked to use them.  Heck, you might even get fired for not using them (think new boss comes in), as is the case at Harrahs.

Despite all this, the most obvious stumbling block is you will take a small hit on the revenue line because you’re not dropping the campaign to the control group.  I can hear it now, “But Jim, I can’t afford to take a hit on the revenue”. 

My answer to this is always the same, “You can’t afford not to take the hit, because you absolutely do not know what your true revenue generation is.”  Imagine being in the position of dramatically understating or overstating the true incremental revenue generated by your campaigns – sometimes for years and years.  This is not a pretty picture when it has to be explained.  Personally, I like to avoid that kind of thing!

So I’m just saying, you might want to mess around with control groups a bit before using them gets forced on you.  Controls are a “best practice”, and I don’t know of anyone that can really defend not using best practices.  If your company has a BI group, it’s only a matter of time before somebody over there forces the use of controls.

So how do you deal with the revenue hit?  Like much of analytics, it’s all about explaining what you are doing and why.  Instead of “gross sales”, the campaign focus becomes “sales per customer” – customer centric, if you will.  You are moving to a more customer-focused measurement system.  The goal is lift, improvement in performance, Marketing Productivity.  The tiny loss in sales from the control group is simply a cost of measuring customer marketing properly. 

And trust me, the insights you will get from using controls will be mind blowing.  You will begin to really understand customer behavior, and that’s the first step to creating truly game-changing customer marketing campaigns.

For example, often the increase in sales attribution to your campaigns from using controls will dwarf the loss in sales by not marketing to controls by a factor of 10 or more.  So while you are worrying about dropping half a percentage in campaign revenue by not using a control, you are leaving an increase of 5% in corrected revenue attribution on the table.

How’s that math working for ya?

Yes, this change will probably will be about as painful as explaining to management why you are moving from measuring hits to measuring page views, but that’s life in analytics.  When there is a better way to measure something, you should embrace it – and teach those around you why it makes more sense to measure that way.

More on the cultural issues of using control groups in the next post.

What about you?  Have you faced this “revenue drop” issue with control groups?  How did you handle it?

Share:  twittergoogle_plusredditlinkedintumblrmail

Follow:  twitterlinkedinrss

Control Group Benefits

This post is part of a series on control groups.  The first post is here, a list of all posts in the series here.

Last time, we finished with a look at halo effects – sales from people who did not respond to the campaign in a way you could track, but did respond to the campaign.   Halo effects are the primary reason your campaign measurements probably underestimate the true response you are generating from e-mail campaigns; the only way to measure these “extra sales” is to use control groups.

But why do these sales happen?  Basic marketing; you created awareness and it resulted in a sale.  For example:

The offer was good for only a week.  People wanted to make a purchase, forgot about it, and remembered after the offer expired.  So they went to the site directly by typing in the URL and made a purchase without the offer. 

This simple story – with probably dozens or hundreds of minor variations – is where the halo effect sales come from.  Your marketing worked, it just did not work the way you thought it would or in a way you could track.

However, if you were measuring success at the Customer level instead of the Campaign level, you would see these sales coming in after the campaign was over by comparing the per customer activity of the test group with the control group.

In other words, Control Groups assign credit to your campaign for:

Brand Execution / Top of Mind

Great Copy / Customized Offer

Timing – right message, at the right time

Engagement Factors

that you can’t measure in any other way.  Measuring Customers instead of Campaigns tells you exactly where the Value was created, and enabling accurate Value measurement allows you to correctly attribute and understand Value creation.

In fact, I could argue that Response metrics are really about your selling process, not the customer buying process – response is not a customer-centric measurement approach.  Response doesn’t take into account the fact that customers will do things the way they want to, not the way you want them to. 

Response is a systems-oriented, mechanical construct that is blind to the effects of emotion.  Control Groups are human-oriented, behavioral constructs that are able to include emotion in the measurement of outcome.  Not that you will know exactly “how” these halo effects occur – you have to let go of that accuracy thing – but a least you will be consistently and completely measuring “what” happened.

And using Control Groups, you can now Measure the un-Measurable – you don’t need “response” to carry the load.  You can measure the ROI of non-response, “feel good” campaigns, for example:

Birthday / anniversary cards – without coupons

“We Love You” / KISS Campaigns

Just calling to see if you need anything

Special Events

These are some of the Highest ROI Best Customer Marketing tactics available.  But people don’t use them, because they can’t figure out how to measure the ROI.  Measurement is simple with Control Groups, and the Marketing folks will love you for using controls because they can actually get back to doing some real Marketing instead of just creating offers all the time. 

That’s boring – for both the Marketers and the customers!

Bullet-proof accountability, simple to explain to management, your friend, the Control Group.  Next time we will get into some operational and cultural issues surrounding the use of Control Groups.

Comments?  Questions?  Have you used control groups, offline or online?  How did it work out for you?

Share:  twittergoogle_plusredditlinkedintumblrmail

Follow:  twitterlinkedinrss

*** Step Up – or Step Back

Information Week gives us this article: Step Up – or Step Back.  Before the Marketers in the audience click Back, I think you should read this article. 

Lead Data: From the annual meeting last month of the Society for Information Management, the percentage of CIOs and other top IT executives reporting directly to CEOs had fallen dramatically from the year-earlier survey, SIM revealed.

The premise is basically this: the “Command and Control” CIO is on the way out; these are the folks that are dropping in rank and no longer reporting to the CEO.  At the same time, we find CIO’s that are business oriented and advocates for process improvement are moving up and more of them are reporting to the CEO.

Makes sense to me.

There seems to be a lot parallels between what is going on with CIO’s and CMO’s; both are looking for a seat at the strategic table.  And both need to become more business-oriented to do it.  I think “business oriented” here is probably just a code word for “more accountable for what you contribute”.  In the case of CMO’s, this includes reaching out into the operational side of the business and finding out how operations affects the success of Marketing.

To go a step further, wanna-be CIO’s and CMO’s not afraid of an accountable orientation would do themselves a huge favor by reaching out to each other; otherwise both or either may be “absorbed into the Network“. 

This pattern playing out over in CIO-land has some lessons for those (mostly analytical) Marketers who aspire to the CMO seat.  If you do aspire to be CMO, read about the CIO’s who do report to the CEO and the business attitude that got them there – the same attitude you need.

Here’s that article link again: Step Up – or Step Back

Share:  twittergoogle_plusredditlinkedintumblrmail

Follow:  twitterlinkedinrss

Why Use Control Groups?

(This post is more or less an narrative from my presentation at the 2007 Washington D.C. eMetrics / Marketing Optimization Summit)

“You know that campaign with the best response rate ever, the one with $5 million in sales?  We lost over $1 million dollars on it, according to Finance.  Something about the difference between Measuring Campaigns and Measuring Customers.”

– Me, giving my boss at HSN a piece of good news, 1991

That, my friends, was the first time I found out just how important control groups are to measuring the success of customer campaigns in an interactive, always on environment. 

The Finance department – through the Business Intelligence unit – was measuring the net profitability of the campaign at the customer level.  We (Marketing) were measuring the net profitability at the campaign level – based on response to the campaign.  The difference was close to $3 million dollars – from a $1.9 million profit using Marketing’s campaign measurement to a nearly $1 million loss using Finance / BI’s customer measurement.

The crux of this difference is always on, self-service demand, or what Kevin calls Organic Demand.  The only way to measure these customer demand effects accurately – and so the true profitability of campaigns – is with control groups.  Online, this issue is primarily relevant to e-mail marketers (customer marketing) but comes into play in lots of different ways – especially so if you have PPC or display advertising taking credit for generating sales from existing customers.

Seems like there is a lot of confusion around what control groups are and why you should care about them, and I’m hoping this post helps to clear some of that up!  But before I lose you in the details, here is why you should care about this topic:

1.  Tactically: First and foremost, if you’re not using control groups, you are most likely chronically underestimating the sales / visits / whatever KPI you generate.  “Response” is almost always lower than actual demand, because your campaigns generate sales / vists / whatever KPI you cannot track through campaign response mechanisms.  Is full credit for what you contribute to the bottom line important to you?  If so, stick around and read the rest of the post.

2.  Strategically: In a multi-platform, multi-channel, multi-source world, control groups are the gold standard in customer campaign measurement.  You will eventually be required to have a common success measurement that can be used for any situation, as opposed to success measurements “customized” for the quirks of every marketing situation that develops.

If you are not using controls, then your campaign results are always suspect.  The fact nobody has asked you yet to prove the sales you claim to generate are actually generated by your campaigns is not an excuse; that day will come.  Will you be ready?  When “prove it” is on the table, the folks using control groups win over those who are not using them every time.

3. Culturally: The concept of “variance reporting” fundamental to the control group idea is very well understood by senior management.  In fact, despite sounding complex, the control group idea is absolutely the easiest to explain to management and generates a tremendous level of confidence in what you are doing. 

This is why confidence in controlled results is so high: there are no “caveats” and no need for specialized understanding from management of different channels or technologies.  No explanations required for technological causes of error – why does this system say sales were this and this other system say sales were that?  No doubts about the source of the ROI, no questions about external effects.  Clean and simple, elegant in execution.

Interested?  OK, here we go.  Here is the idea in a nutshell.

Let’s talk a little about the idea of “incremental”, as in incremental sales or visits.  Incremental means “extra beyond normal” or what is often called “lift” in the database marketing / BI world.  The central issue is this:  if I spend money on a campaign, I want the campaign to generate incremental sales beyond what I would get if I did not do the campaign.  That’s logical, right?  Why else spend the money, if the campaign is not going to lift my sales over and above what they would have been without the campaign?

In offline retail, Wall Street is always after one KPI – called the “comps”, short for “same store sales comparisons”.  What they want to know is for stores open at least a year, what were the sales this quarter versus same quarter last year?  That growth, or lift, is what determines how well the company is doing.  The reason is simple: if they just look at gross demand, it can be inflated by opening new stores.  These new store openings mask the true productivity of the operation, and Wall Street knows productivity is what drives profit growth in retail.  So they want to know the incremental sales versus last year of a finite set of stores open at least a year – not the sales of all stores.  In using this approach, they are controlling for the new store openings – removing the influence of them.

And that’s exactly what control groups are for – to remove the influence of any number of factors, and arrive at the true driver of the incremental change.  

When testing the effectiveness of drugs, one of the control groups is often the placebo – the people who take a sugar pill instead of the real drug.  This is done because of the placebo effect – the tendency of a person to feel better when they are taking a drug.  Why is this done?  Because the testers want to measure the real contribution of the drug – the incremental effects over and above the placebo effect.

OK?  So here is how it works in customer marketing:

1.  Choose a population to target with a campaign

2.  Take out a random sample of that population to use as control – the “control group”.  The remaining members of the population after the sample is taken out are called the “test group”.

3.  Send the campaign to the test group, and do nothing to the control group.  Measure the performance of the test versus control over time, and calculate the incremental impact on the test group of receiving the campaign.

A typical email campaign to best customers might look something like this.  Let’s say the campaign has an end date of 1 week after the drop; the customer has to react within a week to take advantage of the offer:

Control Groups Base Case

Respectable results for a best customer target – you do segment best customers out for different treatment, don’t you?

Here is what the same campaign probably looks like using a control group, after one week of response:

Control Group Static Case

Note that 10% of targets were taken out as control; the remaining 90,000 received the campaign.

If this campaign had dropped to the entire population of 100,000, the campaign that generated $220,000 in sales really generated only $20,000 in sales, because the incremental sales impact of the campaign was only $20,000 ($.20 per e-mail) versus the control group who received no campaign.  The other $200,000 would have been generated by this customer segment without the campaign.  Follow?

Now at this point, you’re probably saying, “Hey Jim, I get it and all that but there’s no fricking way I’m going to implement this at my current job, I mean, I can’t take a hit like that in performance!” 

To which I would say:

1.  Don’t use controls until you change jobs – you’ll look like a major scientific testing hero at your next job!

2.  You don’t have all the data to make this call yet…we need to talk about what I call “halo effects”.

Halo effects are generally the unintended actions taken by the targets of the campaign.  At a basic level, it’s sales generated because of the campaign that you can’t track back to the campaign using a “campaign response” methodology.

Here’s what this campaign looks like after 6 weeks, when probably almost all the the halo effects would be included.  The numbers for each week are cumulative, they include the sales from the prior weeks:

Control Groups Dynamic Case

Now that’s more like it!  If this campaign dropped to the entire population (including the control), it would have generated $295,000.

In this case, there were $75,000 in sales over and above what a “response” measurement of $220,000 shows.  These sales are coming primarily from people who did not respond to the campaign in a way you could track, but did respond to the campaign. 

We’ll dive deeper into explaining how and why this happens, plus address some of the execution and cultural aspects of using control groups in the next post.

Until then, Questions, Comments, Clarifications?


Share:  twittergoogle_plusredditlinkedintumblrmail

Follow:  twitterlinkedinrss