Control Group Benefits

This post is part of a series on control groups.  The first post is here, a list of all posts in the series here.

Last time, we finished with a look at halo effects - sales from people who did not respond to the campaign in a way you could track, but did respond to the campaign.   Halo effects are the primary reason your campaign measurements probably underestimate the true response you are generating from e-mail campaigns; the only way to measure these “extra sales” is to use control groups.

But why do these sales happen?  Basic marketing; you created awareness and it resulted in a sale.  For example:

The offer was good for only a week.  People wanted to make a purchase, forgot about it, and remembered after the offer expired.  So they went to the site directly by typing in the URL and made a purchase without the offer. 

This simple story – with probably dozens or hundreds of minor variations – is where the halo effect sales come from.  Your marketing worked, it just did not work the way you thought it would or in a way you could track.

However, if you were measuring success at the Customer level instead of the Campaign level, you would see these sales coming in after the campaign was over by comparing the per customer activity of the test group with the control group.

In other words, Control Groups assign credit to your campaign for:

Brand Execution / Top of Mind

Great Copy / Customized Offer

Timing – right message, at the right time

Engagement Factors

that you can’t measure in any other way.  Measuring Customers instead of Campaigns tells you exactly where the Value was created, and enabling accurate Value measurement allows you to correctly attribute and understand Value creation.

In fact, I could argue that Response metrics are really about your selling process, not the customer buying process – response is not a customer-centric measurement approach.  Response doesn’t take into account the fact that customers will do things the way they want to, not the way you want them to. 

Response is a systems-oriented, mechanical construct that is blind to the effects of emotion.  Control Groups are human-oriented, behavioral constructs that are able to include emotion in the measurement of outcome.  Not that you will know exactly “how” these halo effects occur - you have to let go of that accuracy thing – but a least you will be consistently and completely measuring ”what” happened.

And using Control Groups, you can now Measure the un-Measurable – you don’t need “response” to carry the load.  You can measure the ROI of non-response, “feel good” campaigns, for example:

Birthday / anniversary cards – without coupons

“We Love You” / KISS Campaigns

Just calling to see if you need anything

Special Events

These are some of the Highest ROI Best Customer Marketing tactics available.  But people don’t use them, because they can’t figure out how to measure the ROI.  Measurement is simple with Control Groups, and the Marketing folks will love you for using controls because they can actually get back to doing some real Marketing instead of just creating offers all the time. 

That’s boring – for both the Marketers and the customers!

Bullet-proof accountability, simple to explain to management, your friend, the Control Group.  Next time we will get into some operational and cultural issues surrounding the use of Control Groups.

Comments?  Questions?  Have you used control groups, offline or online?  How did it work out for you?

3 Responses to “Control Group Benefits”

  1. Brilliant set of posts. The posts definitely gave me a fresh perspective on your segmentation modelling techniques and how to measure campaigns vs. customers. I have to admit I really didn’t connect with the explanation of halo effects when I read the Drilling Down book the first time. These two posts made that concept much more concrete for me. Thanks.

  2. Jim Novo says:

    Thanks for the comment Steve! Glad the posts were helpful and looks like I should include this material in the next edition of the book…

    Retail stores have always had this problem – for the most part, they can’t tell whether demand is coming from something they did or organically. Where possible, Marketers / Analysts should try to uncover the true effect of their programs – the incremental demand over organic (control).

    For the web, e-mail is a very logical / easiest place to start. My experience, the data uncovered using controls often leads to significant and dramatic customer insight – especially in the so far poorly formed area of “engagement”.

Leave a Reply