Category Archives: Driller Q & A

Increase Profit Using Customer State

Jim answers questions from fellow Drillers
(More questions with answers here, Work Overview here, Index of concepts here)


Q: We’ve been playing around with Recency / Frequency scoring in our customer email campaigns as described in your book.  To start, we’re targeting best customers who have stopped interacting with us.  I have just completed a piece of analysis that shows after one of these targeted emails:

1. Purchasers increased 22.9%
2. Transactions increased 69%
3. Revenue increased 71%

A: There you go!

Q: My concern is that what I am seeing is merely a seasonal effect – our revenue peaks in July and August.  So what I should have done is use a control group as you described in the book – which is what I am doing for the October Email.

A: Yep, that’s exactly what control groups are for – to strain out the noise of seasonality, other promotions, etc.  But don’t beat yourself up over it, nothing wrong with poking around and trying to figure out where the levers are first.

Q: Two questions:

1.  What statistical test do I use to demonstrate that the observed changes are not down to chance

2.  How big should my control group be – typically our cohort is 500-800 individuals

A: Good questions…

Continue reading Increase Profit Using Customer State

Optimizing for Customer Value

Jim answers questions from fellow Drillers
(More questions with answers here, Work Overview here, Index of concepts here)


Q: Thank you for creating this useful website!

A: You’re welcome!

Q: When figuring out retention rate for an annual or a 8 months life time cycle period, how do I pick the starting period?  Do I look at their first orders on a date?  Or I pick a time frame such as one month?

A: It depends on:

1. What kind of “retention” you are talking about, the definition, which is probably impacted by the audience for the data

2.  What you will do with the retention data, what kind of decisions will be made and actions be taken because of the data

You should always ask these questions above  when someone requests “retention data” – or any other kind of analysis, for that matter!  For example, there probably is a huge difference in what you would provide to the Board of Directors for an annual benchmark and what you would provide to Marketing people for executing campaigns.

Continue reading Optimizing for Customer Value

When Does a Visitor Need a Coupon? And the Reverse (Incremental Sales / Subsidy Costs)

Jim answers questions from fellow Drillers
(More questions with answers here, Work Overview here, Index of concepts here)


Q: First off, I very much appreciate you sharing all this wonderful content on your blog and conferences such as eMetrics.

A: Thanks for that!

Q: My question is a simple one, but I think the answer may be hard: When does a visitor “need” a coupon?  *Need* defined as: visitor would not have placed an order unless presented with the coupon.

A: Hmmm…methinks we’re going to have to define a few concepts and be clear on the goals to make sure we are nailing this down… visitor versus customer, sales versus profit, etc.  In other words, answer is not hard, but could be complex without defining context.

Q: It’s still a mystery to me why so many retailers seem more than willing to hand over all their margins to Groupon or give coupons to basically all visitors.  I am curious whether you would approach this question using observational data (eg web analytics) or experiments (eg AB testing), or both.

A: Right – is a mystery to me too!

There are certain situations where this approach might be appropriate, but the problem with much web “marketing” (which often is really just advertising without much thought about marketing) is often there is success in a narrow or special situation.  Then the pundits jump on and say “if you’re not doing this you are stupid”, regardless of the business situation and / or without recognizing the special circumstances that are driving success.  This is all the real Marketing stuff people leave out; understanding why it works, under what circumstances, for which segments, involving which products.

Continue reading When Does a Visitor Need a Coupon? And the Reverse (Incremental Sales / Subsidy Costs)