Category Archives: Measuring Engagement

Is Your Digital Budget Big Enough?

Analytical Culture, Digital Analytics, Measuring Engagement

At a high level, 2014 has been a year of questioning the productivity of digital marketing and related measurement of success.  For example, the most frequent C-level complaint about digital is not having a clear understanding of bottom-line digital impact. For background on this topic, see articles here, here, and here.

I’d guess this general view probably has not been helped by the trade reporting on widespread problems in digital ad delivery and accountability systems, where (depending on who you ask) up to 60% of delivered “impressions” were likely fraudulent in one way or another.  People have commented on this problem for years; why it took so long for the industry as a whole to fess up and start taking action on this is an interesting question!

If the trends above continue to play out, over the next 5 years or so we may expect increasing management focus on more accurately defining the contribution of digital – as long as management thinks digital is important to the future of the business.

If the people running companies are having a hard time determining the value of digital to their business, the next logical thought is marketers / analysts probably need to do a better job demonstrating these linkages, yes?  Along those lines, I think it would be helpful for both digital marketers and marketing analytics folks to spend some time this year thinking about and working through two of the primary issues driving this situation:

1.  Got Causation?  How success is measured

In the early days of digital, many people loved quoting the number of “hits” as a success measure.  It took a surprisingly long time to convince these same people the number of files downloaded during a page view did not predict business success ;)

Today, we’re pretty good at finding actions that correlate with specific business metrics like visits or sales, but as the old saying goes, correlation does not imply causation.

If we move to a more causal and demonstrable success measurement system, one of the first ideas you will encounter, particularly if there are some serious data scientists around, in the idea of incremental impact or lift.  This model is the gold standard for determining cause in much of the scientific community.  Personally, I don’t see why with all the data we have access to now, this type of testing is not more widely embraced in digital.

Continue reading Is Your Digital Budget Big Enough? →

Defining Behavioral Segments

DataBase Marketing, Driller Q & A, Measuring Engagement, Newsletters

Jim answers questions from fellow Drillers
(More questions with answers here, Work Overview here, Index of concepts here)

Q: I purchased your book and have a few questions you can hopefully help me out with.

A: Thanks for that, and sure!

Q: We have 4 product lines and 2 of them are seasonal. i.e we have customers that year in year out purchase these items consistently but seasonally, for example, every spring and summer.   Then they are dormant for Fall and Winter.  Should I include these customers along with everyone else when doing an RFM segmentation?

A: Well, it kind of depends what you will using the RF(M) model for, what kinds of marketing programs will be activated by using the scores. If you know you have seasonal customers and their habit is to buy each year, AND you wish to aim retention or reactivation programs at them, I would be tempted to divide the customer base so that seasonal customers are their own segment.  Then run two RF(M)  models – one for the seasonals, and one for everyone else.

Q: If I include seasonal customers, and I run RFM say on a monthly basis, these seasonal customers will climb / fall drastically with time depending on the season, so it seems like it may complicate the scoring process.

A: Sure, and you could segment as I said above.  Or, you could run across a longer time frame, say across 2 – 3 years worth of data. This would “normalize” the two segments into one and take account of the seasonality in the scoring – perhaps be more representative of the business model.  However, the scores would become less sensitive due to the long time frame so the actions of customers less accurately predicted by the model.

Q: Can you provide me with some examples as to how segmentation is carried out?  Let’s say I being with RFM and all my customers are rated 5-5, 5-4, 4-5 etc.  What are the next steps, do we overlay with other characteristics like age, gender, etc?  Or are the 5-3 etc. our actual segments?

A: This goes back to what you want to use the RF(M) model for.  In the standard usage, each score will have roughly the same number of customers in it, those with higher scores will be more likely to respond to marketing and purchase, lower scores less likely.

Continue reading Defining Behavioral Segments →

Increase Profit Using Customer State

DataBase Marketing, Driller Q & A, Measuring Engagement, Newsletters, Web Analytics

Jim answers questions from fellow Drillers
(More questions with answers here, Work Overview here, Index of concepts here)

Q: We’ve been playing around with Recency / Frequency scoring in our customer email campaigns as described in your book.  To start, we’re targeting best customers who have stopped interacting with us.  I have just completed a piece of analysis that shows after one of these targeted emails:

1. Purchasers increased 22.9%
2. Transactions increased 69%
3. Revenue increased 71%

A: There you go!

Q: My concern is that what I am seeing is merely a seasonal effect – our revenue peaks in July and August.  So what I should have done is use a control group as you described in the book – which is what I am doing for the October Email.

A: Yep, that’s exactly what control groups are for – to strain out the noise of seasonality, other promotions, etc.  But don’t beat yourself up over it, nothing wrong with poking around and trying to figure out where the levers are first.

Q: Two questions:

1.  What statistical test do I use to demonstrate that the observed changes are not down to chance

2.  How big should my control group be – typically our cohort is 500-800 individuals

A: Good questions…

Continue reading Increase Profit Using Customer State →