Every year just before the holiday season we take a look at the customer database for the Lab Store – the online retail biz my wife runs – and see what’s up with 10x or more buyers.
I often prefer to look at “worst case” data when doing customer analysis; this way you don’t over-estimate the Potential Value of the business going forward. The beginning of the 4th Quarter is a good time to do this since “holiday” really hasn’t kicked in yet, so you don’t have those buying influences skewing the natural activity in the customer database.
At the end of September, we took a look at all customers, no matter when they became customers, who have purchased from us at least 10 times – a best customer analysis. Considering a “year” to be 9/30 to 9/29, we bucketed them by when their last purchase was – past year, 2 years ago, 3 years ago, 4 or more years ago (the business started 5 years ago).
Here are the results:
Last purchase date was in Percentage of all 10x or more Buyers
9/30/07 – 9/29/08 75%
9/30/06 – 9/29/07 12%
9/30/05 – 9/29/06 8%
9/30/04 – 9/29/05 5%
If this data is still confusing, the second line above would read, “Of all customers who have ever bought 10x or more, 12% last purchased in the period 9/30/06 – 9/29/07.
75% of the best, most productive customers (all else equal, like margins, service costs, etc.) we have ever had since the busines started are still active buyers with us.
Of course, this raises the question: What’s up with the rest of them?
When you segment like this, using the actual customer behavior, an analysis of each segment can be quite revealing. What you are seeing is essentially a LifeCycle model, the attrition pattern of these best buyers. No doubt some will come back, especially out of the “last year” 12% group, but here’s the thing – the longer they are inactive, the less likely they are to come back.
So if you’re going to “do something” with this 12% Recent best defector group, the time to do it is now, going into holiday season, because you have buying momentum running with you. That’s another reason we do this analysis at the end of September ;)
To optimize this action plan, you should know something about who these defectors are. And here is where people make a huge mistake: because they’re inactive buyers, or in the case of non-commerce sites, inactive visitors, they are not being surveyed by the usual survey pops.
Let me say that again: arguably the most important segment of the customer base, dis-engaging best customers, lacks voice under most approaches to VOC. You have to reach out proactively to these people and find out what you did wrong; otherwise you will just keep doing it wrong and continue to churn off your best customers / visitors.
A segmentation technique that is very helpful in this kind of analysis is to think in terms of controllable and uncontrollable customer defection. Since the Lab Store is in the animal supplies business, the sale or death of the animal would create an uncontrollable defection – no amount of “Marketing” is going to get that customer back. Other examples of uncontrollable defection would be when a cable or local newspaper customer moves out of the area serviced by the company, or a manufacturer ceases to produce a line and no longer needs parts.
We know from previous surveys of these defectors this uncontrollable population is about 4% a year. So for each time span above, you can knock 4% off and get to the percent controllable defectors. These are the people you really want to know about, because they are actionable. So as far as the 12% “last year” group, it’s now about 8% of best buyers.
The first controllable defection trigger we would look for in the group would be Product. Product drives everything in a commerce model; it’s why you exist as a commerce entity, and is fundamental to the Brand Promise. If you have a product issue, no amount of Service or other drivers will fix the problem. So Product is where you start. Plus, you should have all the data you need, so it’s easier to get started on right away (no lag time).
Did these defecting best customers buy similar products which resulted in their defection? In other words, something about the packaging, pricing, or performance of the product may be defective. This type of analysis is how we discovered and cured the staple food problem.
The product analysis turned up an interesting trend, or non-trend, actually: while in general there were no hot spots among categories or products purchased by these folks, there was a surprising lack of purchases in the Toy category for the past year. My first question upon seeing this: what toys have they bought from us in the past?
The answer: All of them. About 70% of these defectors had purchased all the toys we carried at some point.
Now, there’s several reasons we don’t carry many toys. The community we serve is extremely sensitive to the issue of materials used in products they expose their animals to, and most animal toys don’t make that cut. Then there is our general philosopy on avoiding commodity products and keeping product selection trimmed to what people want to buy.
But clearly, this looked like a specific, nailed down opportunity to put some effort into this area.
To confirm this idea, we sat down and invented a couple of products, contacted a sample of these defected best customers, and showed them what we were thinking in terms of the new toys. This action confirmed two things for us:
1. Indeed, one of the reasons they stopped buying from us is they already owned most of our toys, and their animals were not on our staple diet, so there was no recurring purchase need.
2. They thought the new toys were very cool ideas – ultra safe materials, fair pricing, unique, etc.
So there you have it – our holiday best customer reactivation plan is set. The items are up and we’ll be dropping a promotional e-mail to just these defected best customers with an “insider deal” on the new toys. The sales generated should be almost completely incremental due to the research we did, but we’ll use a control group anyway, just to make sure we didn’t screw something up.
Now, I know a lot of people in this community would be running all kinds of statistical models and such against this defector data looking for the silver bullet, but what’s interesting about this case is:
1. Critically, the “answer” was not something in the data, the answer was NOT in the data – a lack of toy purchase. If you think this through a minute, you will realize a fundamental flaw in much of digital measurement today: due to the nature of the collection methods, digital analytics is ultra-focused on what happens, forgets what stopped happening.
Opportunity: there’s a ton of money to be made by understanding LACK of activity, why it stopped, and how to address it properly. Hint: marketing is usually not root cause and cannot be used to fix the problem, only to reduce or help correct the problem.
2. The segmentation was totally simple to do with a spreadsheet: Frequency cut with Recency dimension. That’s it. Anybody can do that, anybody. No advanced stats needed. Question is, would someone on your team think of approaching best buyer defection this way? Could they know where to look for a hidden problem like this (what data / reporting config), then see and understand the behavioral implications driving the customer?
Sure, there are ways to set up the modeling to find this out, but you would have to come up with the idea of “missing category” before you constructed the model. Then you would have to make a judgement call on whether the 70% “purchased all toys” number was significant based on intimate knowledge of past customer analysis.
These are the kinds of skills Marketing should be bringing to the party, because the Tech side is often so very stuck in binary thinking. To optimize these Marketing systems, someone has to control for the problems caused by behavior chains like:
did not buy->
so did not get surveyed->
so did not uncover “lack of toys” problem
because the Tech side is always focused on what has happened, as opposed to what has not happened.
Might as well be Marketing who takes up this role, since solving “did not happen” problems almost always has a much higher ROI than looking at what did happen. After all, a 4% conversion rate means 96% did not convert.
In solving this “did not happen” problem, we have both increased the potential for business with dis-engaged best customers and corrected a flaw in the system that would have continued to flush best buyers in the future. One campaign, with both Current and Potential Value.
I’m guessing the response rate and ROI on this best defector holiday campaign will be huge. What do you think?
Update: These toys completely sold out – mostly to the customers who had stopped buying. And 180 days later, about 25% of these “dead” best customers are still 90-day active. The return on investment for this effort is essentially infinity at this point…Follow:
5 thoughts on “Lab Store: Frequent Buyer Analysis”
I have to admit, Jim, I love these stories about the Lab Store. I’m sure I’ve read every one since you started posting about it last year.
In the case of the offer to these customers, are you planning on testing various incentive levels or are you going with a particular incentive that has proved effective in the past with this type of customer due to the size of the population group?
Thanks for the feedback on the Lab Store pieces, I will try to do more of them this next year, I think. Not sure people will be much in the mood for philosophical arguments about what’s wrong with online marketing…
Great question. The first time I test anything, I am much more interested in the quality and stability of the customer reaction than trying to finesse the ROI. Mainly, were we right or wrong about the hypothesis?
One of the testing mantras I have learned over the years is this: there is too little time to *properly* test all the ideas you have, so if an idea is going to fail, fail early (sometimes spectacularly). In other words, I want a rock-solid read on this approach, not a “maybe”.
Also, as I think you alluded to, the population is fairly small, but at the same time potentially powerful. So I want to load this up on the offer side, and I want the offer consistent across the entire population so I can analyze it on the back end without worrying about front-end distortions in behavior due to various shades of offers.
After the test, I want to compare responders with non-responders and try to detect any differences in their past behavior without trying to divine the influence of the different offers on response behavior at the same time. Don’t want too many inputs, want to isolate cause and effect.
This is how I get to sub-segments (if any), which is a much more important idea longer term than tweaking ROI because sub-segments mean I’m into prediction. Once I fully understand what I’m working with and understand the long-term implications, then I can go back and tweak short-term ROI.
So I want dead-on certainty with the results of the first test.
Helping with this idea is the star of the show will be products we invented that are easy to manufacture and have very high margins. So we can do a substantial offer and still make good money. But what I am really most interested in is any repeat behavior *after* the offer, a true re-engagement of these valuable customers with the business.
Copy / offer, something along the idea of a introductory price for our best customers on a brand new toy product line, a secret 30% off sale. Possibly with a “we want your feedback on this toy line, so we’re practically giving them away” kind of approach.
This kind of language, asking for customer feedback on our new products through an introductory offer, is not something that is a new idea for the audience – and we were doing it waaaaay before anything called social media was around :0
But this is a defected segment, so I’m a bit concerned about being too coy with them; I have not completely decided on that. Perhaps better to keep the copy as simple as the structure of the test…
A “bailout package for your critter toy budget”, perhaps?
Great question. There really should be (in my mind, anyway) a huge difference in approach when you are testing a completely new segmentation for the first time. You already know you can tune up ROI later, so why mess up a nice clean read – maybe even get a false read – by trying to make the first attempt out of the box too complex?
Load them up and fire all the guns you’ve got.
Any opinions on the copy approach we should use? What further simple analysis might tighten the segmentation and result in better overall customer experience with this campaign?
Clues have been provided…any Sherlocks out there?
Our of curiosity, how are you tracking all this? What are you using for the testing?
P.S. I’ve only been reading your blog for 2 months, so I apologize if this is explained elsewhere.
Hi Burton, thanks for the comment and welcome!
Not sure if you mean the questions above literally, but I will assume so. If I don’t answer your question, feel free to ask again…
It’s very simple – we do it the old fashioned way – by hand, with a MS Access Database and sometimes and Excel spreadsheet. Our backend processing system – pick, pack and ship – creates the customer database, aggregating orders and providing high level KPI’s. More on that software here.
As far as “what are we using for testing”, not sure what you’re driving at there, we’re not using any fancy software or anything. We know who the target is for the promotion, we export the e-mails from the database, drop the e-mails, query the database at 30-60-90 days after drop to find out how many of the target audience responded, what they spent, etc.
The idea of the 30-60-90 is we don’t care as much what the initial response was, but what the behavior looks like played out over time – did these defected best customers become active customers again, did we address the reason they stopped buying effectively?
That’s the long-term, higher value issue we are trying to address, not whether we can get them to respond with a discounted offer – people will buy anything if you cut the price enough!
Hope that answered your question, if not, try again…
Yep, you answered my questions, thanks!