Monthly Archives: March 2007

Measuring Customer Experience ROMI #2: Lab Store – New Customer Kits

Analytics Education, DataBase Marketing, Lab Store, Marketing thru Operations

Here’s another Customer Experience kind of test that proves you can generate incremental profit by improving the Experience.  You just have to make sure customers want the experience “improved”.  This example is from the Lab Store and the ROMI on this little program is a real eye popper.

Back in the old days (meaning the 80’s), what I guess is now called WOW was referred to as “surprise and delight”.  Essentially, this 2-step idea works like this: when you surprise the customer, you really get their attention.  If you can get their attention by surprise and delight them at the same time (instead of pissing them off with your surprise), then you are going to have a more loyal customer.  The trick, of course, is to somehow make more money doing it…

New Customer Kits are a very simple way to do this, and in my remote retailing experience, it works every time.  First impressions, in case you didn’t know, are really important – and especially so in remote retailing, where there is no way for the customer to get any tangible “feeling” for the company.  Sure, you have copy on the web site that paints a picture.  But how many times have people read all this wonderful copy only to be screwed when delivered the tangible experience?

The challenge is to design a kit that is relatively inexpensive yet packs an emotional delight.  Lots of people toss extra stuff for the customer in the first order, but that stuff is usually company-centric, for example, “Here is a magnet with our URL on it” or “Here is a catalog of our other products”.  That’s fine, but it’s neither surprising nor delightful.

Here is what makes up a good New Customer Kit, based on years of testing:

1.  A letter or other message from the company that Welcomes the customer, talks about the people and philosophy behind the company, and reinforces any guarantees or promises that are part of the Brand.  This piece must be written carefully, and from a customer-centric point of view.  No “we we” stuff.

2.  A free gift.  This gift must be related to the merchandise or general category being purchased, and must not be discards, seconds, or defective merch.  Giving a new customer something that is dented or discolored is not a gift, it’s an insult.  Giving a new customer something that is promotional (magnet) may be a gift, but it is expected and not particularly delightful.  Giving a new customer a “gift” because they made a first purchase (Buy today and we’ll include a…) might be delightful but sure is not surprising.  Ignore the above cautions at your own peril.

3.  Free Samples, if relevant to the business.  Anything that is consumable and generates repeat purchase is ideal.

Anyway, I suppose you’re expecting some kind of numbers to go along with all the fuzzy-wuzzy “Oh, if we just make their experience better, they will be more loyal” drivel you hear all the time online.  This is the Marketing Productivity Blog, after all, right?  OK, here are the stats on this technique from the Lab Store.  As usual, this promotion was tested versus control (new customers who did not receive a New Customer Kit are control) and we compare sales activity of both test and control over the next 90 days.  Why 90 days?  Well, if it makes money at 90 days, it sure makes money at 120…

Average cost of New Member Kit (there are several versions) – $.74

Increase in 90-day second purchase rate, test versus control – over 30%

90-day ROMI – 4,891%  ($36.68 in net profit for every $.75 spent)

Surprised and Delighted Customers – Priceless

Now that the bottom line has been presented, the black box folks simply interested in the “what happens” can skip the next part.  If you want to know why it works and maybe learn something useful you can port elsewhere, read on.

New Customer Kits are a great way to shape Theatre of the Mind. 

What you have with a remote retailing customer is a “theatre of the mind” scenario, much like you have in radio advertising.  Customers can’t see or touch you, so “Cues” become extremely important; if you don’t populate the theatre of the mind for the customer, the customer will go ahead and populate it themselves.  If you want some control over the image of your company people create in their head, you need to be proactive.  Theatre of the mind, folks.  Very powerful stuff. 

Our New Customer Kit generates absolutely tons of “Thank You” e-mails from new customers who want to tell us all about how great the experience was purchasing from the Lab Store.  Now, I think you’d agree that purchasing from a web site isn’t a particularly thrilling experience in any way, but if you really listen (and understand a bit of Consumer Psychology) these customers are not really talking about the web site, or even our company.  

What they really are saying is they are very happy with themselves for making a first purchase from us; our actions have confirmed they made a good decision.  Remember, this is remote retailing.  There is risk to the customer, especially on that first purchase; they have no idea if their expectations based on the web site copy are going to match the reality of delivery.  They are concerned about what might happen – will they be proven smart or dumb for taking this risk?

When we deliver the products they ordered in a timely way we meet expectations.  When we deliver these products carefully packed in a pristine new box packed with fresh blank newspaper, we probably exceed expectations by a bit.  But when these new customers get to the Welcome letter, the free gift, and the samples, we blow out their expectations. 

The picture these new customers had in their mind of our company based on the web site experience is then permanently altered; we’re doing brain surgery for 74 cents a head.

Now, I have a question for you – is this program Marketing or Customer Experience Management?

Measuring Customer Experience ROMI #1: Nice to New Customers

Analytical Culture, DataBase Marketing, Marketing thru Operations

I’m going to preface this piece by saying I don’t really think “Customer Experience Management” is anything different from smart, integrated Marketing and Customer Service.  If there isn’t an actionable framework for it, like Ron, I’m not sure CEM has a future, other than to create something for people to talk about, and maybe sell some software…

Whichever direction you believe in, here is an interesting case that makes several points about this area of discussion.

The Nice to New Customers test was conducted at Home Shopping Network in 1994.  The idea came from the annual survey of all customers that indicated that the “average” customer felt the “new customer experience” was “as expected”.  Given the high percentage of 1x buyers we were experiencing (as do all interactive remote retailers), I thought, “Hmm, maybe if we deliver a customized first purchase experience and process, these new customers will be more likely to make a second purchase”.  Sounds logical, right?  This was a Business SWAT case since it involved Marketing, Customer Service, IT, and Telecommunications, all working together to set it up, determine the metrics, make sure Management understood the impact of the test on existing silo Scorecards, etc.  In other words, I sold my soul to get this test to happen.

We set up a pretty elaborate test where a random sample of new customers (about 100,000, a solid test group) were shunted to our “best agents” and given a new “Welcome Treatment”.  Instead of the general “get them off the phone as fast as you can” attitude prevalent in the network, these reps had permission to spend as much time with the customer as the customer wanted and generally customize the experience.  There was a lot of role play and monitoring connected to this effort, and the service managers on the project were convinced these new customers were in fact treated to a much better initial experience than the average new customer.  In fact, the customers seemed thrilled.  So far, so good. 

Problem was, this test group of new customers exposed to a better “Customer Experience” ended up generating no incremental sales versus control.  Well, there you go.  We lost a ton of money on this test, a stellar -118% ROMI, because we literally had to pay back customer service out of the marketing budget for the lost productivity in the network due to the test.  Hey, that was the deal I cut to get this test done.  You win some, you lose some.

But it gets worse.  When we started dicing the post-analysis of the test down to behavioral groups based on the details of the first transaction, we found there was actually some incremental sales lift among new customers with “light buyer” initial profiles.  This is good.  Problem was (and you know what is coming, don’t you?), new customers with heavy buyer profiles were negatively impacted, and because the Potential Value of this group was so huge, the losses versus control in this relatively small number of folks far outweighed the gains in light buyers, causing the net effect of the promotion to be negative.

Isn’t that a fine kettle of fish?  Being Nice to potential Best Customers killed the test.

When we surveyed these customers in the test after we knew their behavioral profiles (to make sure we knew the behavioral context of their answers) they basically told us this: they were expecting a very operationally efficient transaction and we provided them a customer-centric one.  Cognitively, they were making an impulse purchase and they wanted an impulse transaction, not an empathetic one.  This disconnect caused post-purchase dissonance and reduced intent to purchase.  Using today’s language, we were basically “spamming” them; we were overstepping any Permission we had to engage them at a more personal level.  And this negative effect was most pronounced among new customers with high Potential Value.  In hindsight, knowing what we knew about the psychological profile of Best Buyers, this made all the sense in the world and was an interesting confirmation of the test results.

The CFO, well, he didn’t think this result was so interesting…but did applaud the idea that we would step up to the plate and actually pay back customer service for the losses related to decreased productivity in the network out of the Marketing budget.  It was the first time anybody had done this kind of intra-silo payment and really paved the way for tighter integration between Marketing and Service.

You might consider this test result when evaluating your e-mail contact strategy, at least for new customers.  Are you sure you are generating maximum revenue?  What if the half percent or so that unsubscribe each month are future Best Customers with high Potential Value?  Do you use control groups, do you know the answer to this question?

Interactive behavior provides a very special backdrop for Marketing and Service; be careful what you ask for. 

I’m not saying if you did this test you would get the same results.  What I am saying is you cannot assume all the stuff you read about “Customer Experience” online is going to work with your customers.  You simply have to test these ideas with real customers and measure the results.  And if you are dealing with interactive customers, keep in mind that “Customer in Control” is something you might not want to mess with.  In other words, sometimes Control is the Experience, particularly if the general Marketing / Brand backdrop is Operational Efficiency.

It’s one thing to start a company saying you are going to deliver some kind of superior Customer Experience and embed this idea in your service delivery model.  We all know these kinds of companies.  It’s a completely different idea to think that you are going to improve the current experience at your company, and this effort is going to have positive effects for both the customer and the company because it sounds logical to you.

Lessons learned:

 1.  The bottom line lesson here really was about a poorly constructed test based on a faulty customer survey methodology.  Without the customer opinion first tied to an actual behavior, we had no option other than to use the opinion of the “average customer” as a base to act against.  Because of this, the only action we could take was against  “all new customers”, and ended up shooting ourselves in the foot.  Based on the post test dicing, we later retested and found (surprise, surpirse) a program like this could be extremely profitable when we treated targeted new customers differently based on their Potential Value. 

If we had this behavioral information (the initial Light Buyer / Best Buyer profiles) tied to the survey responses from the beginning, we would have understood these segments were different and designed the test accordingly.  Make sure if you are going to take some kind of action on a survey, you first understand a behavior and then survey the people with that behavior.  To do it the other way around, trying to “back into the behavior”, wastes a lot of time and money just in the data gathering and processing itself, never mind in the “re-testing” we had to go through once we knew what was really going on.

2.  It doesn’t always pay out to be Nice to New Customers.  Sometimes they simply want what they expect.

Lab Store: Automating Worst Practices

Analytical Culture, DataBase Marketing, Lab Store

The news that Omniture has acquired Touch Clarity is shaking up the world of web analytics a bit.  Machine automation has always been a very sexy sell for software companies.  The problem is people think it’s a magic bullet and often end up using these tools to their disadvantage because they do not have the experience to really understand how to use the tools properly.  Then they get caught in trap of Reporting versus Analysis.

Here is a real world example from the Lab Store.  I am constantly fighting the Google AdWords A/B split testing algorithm for rotating ads.  Google almost always picks the wrong ad to run more frequently so I have to force it to run 50 / 50 in order to get accurate results.  How do I know Google is picking the wrong ad?  Because I have seen thousands of such tests, online and off, and I have a “feel” for these things based on my background in Database Marketing, Consumer Behavior and Psychology.  In each case where Google has picked one ad over another, and where I have forced it to then run the ads 50 / 50, it ends up I was right – Google picked the ad that generated the least profit per dollar of PPC spend as “best” and demoted the more profitable ad until it was not running at all.

Why does this happen?  Because Google isn’t smart enough to understand the complexity of the customer behavior in the Lab Store - and it can’t be, given the number of clients it has.  If you have done a lot of this kind of testing, you know that often the campaign with the highest response rate generates the lowest quality customers.  While these campaigns were running, I could see that the visitors generated by the campaigns Google picked as “best” were actually inferior to the visitors generated by the campaigns Google demoted, using a variety of metrics other than conversion (primarily Recency).  In other words, I was able to predict Google was doing the wrong thing by looking at the Customer LifeCycle. When I forced Google to run the ads 50 / 50 to give the demoted ads a chance, I was proven right – the campaigns Google demoted had a 90-day ROMI averaging 2.1 times higher than the campaigns Google promoted.

Look, I know these are software companies and their sole purpose in life is to create the next big thing and sell their software into it.  That’s fine, and frankly, I hope they are successful in doing it, because it will create a tremendous amount of business down the road for database marketing consultants as “machine optimization” hits the wall and companies need to be rescued from the results of it.  Just like they had to be rescued from demographic clustering in the 80’s and data mining in the 90’s.

People are always looking for the easy way out, and it ends up costing them more in the long run because they don’t really understand what the tool does and does not do. Perhaps that is simply the state of Marketing today.  So be it…

If you are an analyst and you see a black-box test result that simply does not make any sense based on your past experience, I encourage you to question the result, find a way to test it outside the system.  Learn why, because this kind of incident usually will lead to a shattering of some myth or bias you will be most happy to fully understand!