This post is part of a series on control groups.  The first post is here, a list of all posts in the series here.

Last time, we finished with a look at halo effects – sales from people who did not respond to the campaign in a way you could track, but did respond to the campaign.  Halo effects are the primary reason your campaign measurements probably underestimate the true response you are generating from e-mail campaigns; the only way to measure these “extra sales” is to use control groups.

But why do these sales happen?  Basic marketing; you created awareness and it resulted in a sale.  For example:

The offer was good for only a week.  People wanted to make a purchase, forgot about it, and remembered after the offer expired.  So they went to the site directly by typing in the URL and made a purchase without the offer.

This simple story – with probably dozens or hundreds of minor variations – is where the halo effect sales come from.  Your marketing worked, it just did not work the way you thought it would or in a way you could track.

However, if you were measuring success at the Customer level instead of the Campaign level, you would see these sales coming in after the campaign was over by comparing the per customer activity of the test group with the control group.

In other words, Control Groups assign credit to your campaign for:

Brand Execution / Top of Mind

Great Copy / Customized Offer

Timing – right message, at the right time

Engagement Factors

that you can’t measure in any other way.  Measuring Customers instead of Campaigns tells you exactly where the Value was created, and enabling accurate Value measurement allows you to correctly attribute and understand Value creation.

In fact, I could argue that Response metrics are really about your selling process, not theÂcustomer buying process – response is not a customer-centric measurement approach. Response doesn’t take into account the fact that customers will do things the way they want to, not the way you want them to.

Response is a systems-oriented, mechanical construct that is blind to the effects of emotion.  Control Groups are human-oriented, behavioral constructs that are able to include emotion in the measurement of outcome.  Not that you will know exactly “how” these halo effects occur – you have to let go of that accuracy thing – but a least you will be consistently and completely measuring “what” happened.

And using Control Groups, you can now Measure the un-Measurable -“ you don’t need “response” to carry the load.  You can measure the ROI of non-response, “feel good” campaigns, for example:

Birthday / anniversary cards – without coupons

“We Love You” / KISS Campaigns

Just calling to see if you need anything

Special Events

These are some of the Highest ROI Best Customer Marketing tactics available.  But people don’t use them, because they can’t figure out how to measure the ROI. Measurement is simple with Control Groups, and the Marketing folks will love you for using controls because they can actually get back to doing some real Marketing instead of just creating offers all the time.

That’s boring – for both the Marketers and the customers!

Bullet-proof accountability, simple to explain to management, your friend, the Control Group.  Next time we will get into some operational and cultural issues surrounding the use of Control Groups.

Comments?  Questions? Have you used control groups, offline or online?  How did it work out for you?