Yep, you answered my questions, thanks!

Burton

Not sure if you mean the questions above literally, but I will assume so. If I don’t answer your question, feel free to ask again…

It’s very simple – we do it the old fashioned way – by hand, with a MS Access Database and sometimes and Excel spreadsheet. Our backend processing system – pick, pack and ship – creates the customer database, aggregating orders and providing high level KPI’s. More on that software here.

As far as “what are we using for testing”, not sure what you’re driving at there, we’re not using any fancy software or anything. We know who the target is for the promotion, we export the e-mails from the database, drop the e-mails, query the database at 30-60-90 days after drop to find out how many of the target audience responded, what they spent, etc.

The idea of the 30-60-90 is we don’t care as much what the initial response was, but what the behavior looks like played out over time – did these defected best customers become active customers again, did we address the reason they stopped buying effectively?

That’s the long-term, higher value issue we are trying to address, not whether we can get them to respond with a discounted offer – people will buy anything if you cut the price enough!

Hope that answered your question, if not, try again…

Our of curiosity, how are you tracking all this? What are you using for the testing?

Thanks.

Burton

P.S. I’ve only been reading your blog for 2 months, so I apologize if this is explained elsewhere.

Great question. The first time I test anything, I am much more interested in the quality and stability of the customer reaction than trying to finesse the ROI. Mainly, were we right or wrong about the hypothesis?

One of the testing mantras I have learned over the years is this: there is too little time to *properly* test all the ideas you have, so if an idea is going to fail, fail early (sometimes spectacularly). In other words, I want a rock-solid read on this approach, not a “maybe”.

Also, as I think you alluded to, the population is fairly small, but at the same time potentially powerful. So I want to load this up on the offer side, and I want the offer consistent across the entire population so I can analyze it on the back end without worrying about front-end distortions in behavior due to various shades of offers.

After the test, I want to compare responders with non-responders and try to detect any differences in their past behavior without trying to divine the influence of the different offers on response behavior at the same time. Don’t want too many inputs, want to isolate cause and effect.

This is how I get to sub-segments (if any), which is a much more important idea longer term than tweaking ROI because sub-segments mean I’m into prediction. Once I fully understand what I’m working with and understand the long-term implications, then I can go back and tweak short-term ROI.

So I want dead-on certainty with the results of the first test.

Helping with this idea is the star of the show will be products we invented that are easy to manufacture and have very high margins. So we can do a substantial offer and still make good money. But what I am really most interested in is any repeat behavior *after* the offer, a true re-engagement of these valuable customers with the business.

Copy / offer, something along the idea of a introductory price for our best customers on a brand new toy product line, a secret 30% off sale. Possibly with a “we want your feedback on this toy line, so we’re practically giving them away” kind of approach.

This kind of language, asking for customer feedback on our new products through an introductory offer, is not something that is a new idea for the audience – and we were doing it waaaaay before anything called social media was around :0

But this is a defected segment, so I’m a bit concerned about being too coy with them; I have not completely decided on that. Perhaps better to keep the copy as simple as the structure of the test…

A “bailout package for your critter toy budget”, perhaps?

Great question. There really should be (in my mind, anyway) a huge difference in approach when you are testing a completely new segmentation for the first time. You already know you can tune up ROI later, so why mess up a nice clean read – maybe even get a false read – by trying to make the first attempt out of the box too complex?

Load them up and fire all the guns you’ve got.

Any opinions on the copy approach we should use? What further simple analysis might tighten the segmentation and result in better overall customer experience with this campaign?

Clues have been provided…any Sherlocks out there?

In the case of the offer to these customers, are you planning on testing various incentive levels or are you going with a particular incentive that has proved effective in the past with this type of customer due to the size of the population group?