Jim answers questions from fellow Drillers
(More questions with answers here, Work Overview here, Index of concepts here)

Q: We’ve been playing around with Recency / Frequency scoring in our customer email campaigns as described in your book.  To start, we’re targeting best customers who have stopped interacting with us.  I have just completed a piece of analysis that shows after one of these targeted emails:

1. Purchasers increased 22.9%
2. Transactions increased 69%
3. Revenue increased 71%

A: There you go!

Q: My concern is that what I am seeing is merely a seasonal effect – our revenue peaks in July and August.  So what I should have done is use a control group as you described in the book – which is what I am doing for the October Email.

A: Yep, that’s exactly what control groups are for – to strain out the noise of seasonality, other promotions, etc.  But don’t beat yourself up over it, nothing wrong with poking around and trying to figure out where the levers are first.

Q: Two questions:

1.  What statistical test do I use to demonstrate that the observed changes are not down to chance

2.  How big should my control group be – typically our cohort is 500-800 individuals

A: Good questions…

Jim answers questions from fellow Drillers
(More questions with answers here, Work Overview here, Index of concepts here)

Q: I was wondering if you’ve done any work with, or given thought to, companies who have a cloud based Freemium business model?

Should they be tracking usage (or anything) at the free level?  Should they be tracking usage at the paid level?  I’m sure defection rates are a big problem, but I’m wondering how many focus on engagement thru mass marketing versus trying to keep what they’ve got, or influence the free users to make the leap to paid.  Any thoughts on this?  Maybe you could do a blog post on it.  It seems like a good fit with your brand of analysis but I’m just starting to think it through…

A: I just finished an analysis that’s a good example of this problem.  Behavior during the Freemium period can predict who is highly likely to become a paying customer, who will need marketing efforts like additional sampling / package discounts, and who will not become a customer no matter what you do.