Riffing off a great post by George on marketing measurement, here’s a very specific example of how Marketers have to think differently when they are dealing with interactive environments, from my days at HSN.

We spent about 5 years and $100 million dollars trying to prove offline media would drive new customer acquisition and sales.  We tried everything.  Billboards.  TV.  Radio.  Newspapers.  TV Guides – local, national, and cable.  Flyers,  Shoppers, FSI’s.  Spot cable.  All of it, in just about every combination you can think of.

Each time we did these tests, we set up control markets and looked for Incremental sales in the media markets versus those with no media, based on revenue per household.  We found incremental sales in just about every case. 

The problem was this: even though the media created incremental sales, these sales were never enough to pay back the media on a net basis, meaning (roughly) (Gross Margin – Campaign Cost) – Variable Overhead was negative – even when you took into account the LifeTime Value of a new customer.  Even when you looked at the test markets versus control 3 months, 6 months, and 12 months later, for those who might be thinking about “Brand” or “Awareness”.

If you’re thinking perhaps the campaigns were weak or light on exposure, I offer you this: when the campaigns included coupons, the redemptions were absolutely huge.  That’s good, right?

Well, in a word, No.  Think about it.  There was barely any lift in sales at all, yet huge numbers of coupons were redeemed.  Meaning?

This means that virtually all the coupons were redeemed by current customers, and the coupon / response did not change their behavior.  They bought at the same rate as they would have without the coupon.  It means we gave a ton of margin away in addition to the cost of the Campaign, and generated no increase in Sales.  We literally would have been better off (financially) by doing absolutely nothing.

So, we’re talking through all this and thinking, we have to be missing something.  I mean, you just can’t possibly buy a 100 showing in Boards with saturation TRP bombing on radio and 1/2 page ads in the Weds shopper section of the daily newspaper, along with FSI’s on Sunday and get no effect on sales.  That breaks all the rules of Marketing.  You know, Reach and Frequency and all that.  Awareness and so forth.  Branding.

Unless, of course, the rules have changed, for us, for this Interactive thing.  Maybe it’s, you know, different.

OK, so let’s reverse engineer the question.  Let’s find the markets where we have the best revenue per household, and compare them with the markets where we have weak revenue per household, and see if we can find any differences.  You know, demographics and such.  Household income.  Presence of children.  All that stuff.

Nothing.  No correlations.

Then we’re analyzing the results of some spot cable testing, where we were running heavy rotations on the local cable systems carrying the HSN signal, again with test and control systems.  Sometimes, it seemed to make a huge difference.  Other times, nothing.

We boosted investment in the cable testing to try and replicate the positive outcomes.  Nothing.  In fact, the cable systems with above average revenue per household seemed to stay that way, with or without ads.  The poor performers stayed that way, with or without advertising.  You would normally attribute a result like this to demographics.  But we already knew demographics didn’t matter.

OK, what do the high performing systems have in common?  What do the weak performing systems have in common?

Then it hits us.  Channel position.  The high performers tend to carry our signal on a low channel number, the low performers tend to carry us on a high channel number.  But that doesn’t make any sense, what could “channel number” do to affect sales?

Then we get the 2nd smack to the upside.  The major broadcast networks are almost always carried by the cable system on low channel numbers, typically the Frequency (local TV 8 on channel 8).

Oh, man.  $100 million in advertising spend to find this out?

It can’t be.

So, we go down to the cable affiliate area and find some cable systems that might want to play ball with us on a test.  We’ll pay all the costs of them moving our signal from up in the 40’s down to the single digits, where the major networks are carried.  We’ll pay for customer notification letters, new channel cards, all that, if they’ll just try it.

It worked.  Almost every time.  And it made sense.  Think about it.

The HSN TV program itself (the “user interface” for you web analytics folks), the interactive nature of the real time presentation, the games, the bargains, pulls people in, gets them hooked.  But they have to see the program to get hooked, they have to experience the Interactivity of it to really “get it”.  “Advertising” can’t do that, can’t provide or describe what the experience is like.  So we can’t Push them to the signal.  We have to Pull them, and to do that we have to go to where the eyeballs already are – on the major networks.

If we move our channel next to where all the eyeballs are, then we dramatically increase the number of times the average person passes over our channel as they search for entertainment using the remote, which increases the likelihood they will see a product they are interested in, which causes them to watch the show and experience the Interactivity of it.

In other words, we get a higher ranking in the “TV Search Engine” by being close to the broadcast networks.

This play was so consistent, we built a predictive model off the results, and could estimate the increase in profits from doing this for each cable system.  This means we can calculate how much we could spend to make it happen.  The ROI was enormous, just huge.

So we took the Mass Media budget and spent it on cable channel realignments, which involved no Advertising at all.  Because it was the right thing to do – there was no more efficient way to optimize that piece of the Marketing budget for the Company.

Sometimes, Marketing isn’t about Advertising.  It’s about understanding Behavior.  When you add the word “Interactive” to the mix, this Behavior issue becomes much, much more urgent to address.

Will you do the right thing?  Perhaps invest in analyzing customer behavior, so you can allocate your media budget more efficiently?  Maybe try some control groups in your e-mail program?

Or will you buy Advertising just because you have an Ad budget?