Marketing Funnel Not Dead,
Using Funnel Model for Attribution Is

It’s become fashionable to declare the “Marketing Funnel Model” dead.

For example, here is a post worth reading on this topic by Rok Hrastnik.  There are some very good points in this post on why using a funnel to attribute media value is really a troubled idea.  I was flagged on this post because it has a quote from me that seems to support Rok’s thesis about the death of the funnel model and the related idea, “Direct Response Measurement is a Wet Dream”.   The quote is from a comment I made on a post by Avinash where we were discussing the value of sequential attribution models:

There are simply limits on what can be “proven” given various constraints, and that’s where experience and a certain amount of gut feel based on knowledge of customer kick in.  If you can’t measure it properly, just say so. So much damage has been done in this area by creating false confidence, especially around the value of sequential attribution models where people sit around and assign gut values to the steps.  Acting on faulty models is worse than having no information at all.

But none of this means the Funnel Model is dead, or that Direct Response Measurement overall is a Wet Dream.  What’s (hopefully) dead is  people using the funnel model inappropriately for tasks it was never designed for, in this case multi-step attribution of media value to goal achievement.  On the other hand, if this specific funnel use case is what Rok was coming after, I agree, because it didn’t make any sense to use a funnel model for this idea in the first place.

Let’s unpack these ideas

Funnel thinking is based on a relatively reliable model of human behavior, AIDA.  This model from human psychology does not specify tools, channels, or media.  It simply says that there is a path to purchase most humans follow.  That is:

A – Attention: (Awareness): attract the attention of the customer
I – Interest:  (Intent) promote advantages and benefits
D – Desire: convince customers the product will satisfy their needs
A – Action: lead customers towards taking action / purchace

Example:  I’m Aware of tons of products I would never buy.  There are lots of products I think are Interesting but I have no Desire for.  There’s a short list of products I Desire but have not Acted on.  The list of products in my head worthy of purchase consideration gets smaller and smaller at each stage of the AIDA model.  This is the funnel.

The AIDA funnel has not changed and it’s not dead.

It’s a model of human behavior, not media consumption.

Said another way, it’s not the tools, channels, or media that are a funnel, it’s the way humans process buying decisions that is a funnel, left side of chart below.  Media touches and impact, relative to the AIDA path, typically do not happen in a linear or funnel fashion, see right side of chart below, click to enlarge:

In a chaotic environment such as the one above, it’s near impossible to determine the contribution to end goal conversion of any one media touch or step in a multiple-step the sequence; there’s just too many uncontrolled variables.  I guess Rok knows this, because in a follow-up post, he presents his alternative view he calls the purchase cycle:

Yea, I know, looks like a funnel to me too.  In fact, it’s the AIDA model, right?  So, I think what Rok was saying in his original post is the using these funnel ideas for media attribution is screwed up, not the funnel idea itself.  And I would totally agree with that notion.

What Went Wrong

Where I think using a funnel model for attribution purposes starts to go off the rails is confusing measurement of  funnel state / position during  the sales cycle and attributing state to particular media.

These are two completely different ideas.  The first idea is simply a position, state or status with no value implied; it does not really matter “how” someone got to a state, just that they are there.  State tells you about what message might be most effective; where someone is in the process.   The second idea is brave but lacking data; given nothing but  a media touch in a particular sequence, let’s determine what value a touch contributed to end goal.  This idea really has nothing to do with funnel states at all.  Stir in an incomplete and inaccurate picture of all the media touches happening during the sales process (see Rok’s first post), and you have a real mess.

If what an analyst wants to achieve is similar to this second idea, funnel thinking is probably not a great model to use.  The best way to attack this idea is through media mix modeling.  Analysts still won’t be able to see the contribution of specific touches to value creation, but will be able to accurately measure the overall contribution of the different media types used in the mix to outcome.  The actual sequence or level of media exposures at an individual level is still not knowable  but a mix model can determine, in the aggregate, which media types and mixes drive optimal sales or other actions.

The fact many orgs don’t have the tools or institutional stamina to do media mix measurement properly should not result in people declaring they have measurement of these media step effects.  In fact, all they really have is a list of media exposures that is clearly an incomplete and broken picture, and guesses about the value creation at any step.

Hopefully, this approach is dead (or more likely dying) as well.

In fact, the whole topic of  attribution needs  a more complete discussion so analysts and marketers fully understand the benefits and pitfalls of various approaches to attribution and set themselves up for success in the future.  Those interested in this topic from a media / goal conversion perspective should check out my recent post for Econsultancy: Online attribution models: getting close.

By the way, I think this sequential touch data is fabulous for gaining a greater understanding of media interactions, it’s a great step forward in the evolution of digital analytics.  The vendors can’t keep people from abusing this data, and it can be used for some very creative testing ideas, which I cover in the Econsultancy post.

If analysts can’t measure these media contribution to goal effects properly, they should just say so and ask for the resources to measure them correctly.  There’s no shame in saying you just do not have the right tools or enough resources.  If you have to provide metrics, just be honest about what they do and don’t mean.  But please, don’t sit around and assign arbitrary values to exposures based on budgets or whatever else.  A list of events is not a media value measurement, and you hurt the marketing measurement cause when you just make stuff up.  And people wonder why analysts don’t get respect…

What You Can Do

If you have the capability to get a bit more sophisticated but are not ready for mix modeling or controlled testing, here’s a framework for approaching this media value attribution issue.

For Awareness, secure Awareness-generating media and measure the effectiveness of those media at generating Awareness using Awareness metrics.  Forget the rest of the funnel for these Awareness media, the next steps in the AIDA funnel are not the primary success metric of Awareness media.  If you are successful here, some percentage of the audience will move to the Interest stage.

Secure Interest-generating media and measure the effectiveness of those media at generating Interest using Interest metrics.  Forget the rest of the funnel for these Interest media, the next steps in the AIDA funnel are not the primary success metric of Interest media.

And so forth.  Those interested in more on this idea check out the Marketing Bands series here.   A similar approach can be taken for the entire Media / Marketing / Service effort, one that after initial goal achievement adds customer centricity as part of the marketing mission.  The financial results of such an approach are here.

I’m sure some people will say, “But Jim, one piece of media can have effects in different AIDA layers.  For example, a campaign designed to generate Awareness might also create some Interest, Desire, or Action (see the chart above).  How do you account for or measure these spillover effects using your approach?”

The correct answer is a media mix model.  Do you have a media mix model?  No?  Then I’m sorry, you’re just going to have to deal with an imperfect marketing measurement system, and realize there will be beneficial effects you cannot measure.  If what you can measure properly is successful, you can take comfort in knowing the actual results are probably even better!

Yes, the Marketing Bands framework is an imperfect approach, but without a mix model, it’s at least a realistic way to approach this marketing measurement challenge.  And it’s a whole lot better then fabricating attribution out of thin air using a list of sequential exposures, a model that is broken in so many obvious ways if you want to measure campaign effectiveness.

Direct response measurement is not a Dream or Wet for that matter, but it’s important to use it when it makes sense. Direct response metrics don’t take into account  all the vagaries of  media touches, real or imagined.  That’s true, but who said they were supposed to?  The advantage of direct response measurement is consistency and predictability, not accuracy.  If you want to measure Awareness, don’t use a direct response metric, use an Awareness metric.  Problem is, measuring Awareness properly is not easy and it’s expensive.  So sure, conversion has always been a shallow metric, but if it’s all you can afford, it’s pretty important.  Cost per order or campaign, preferred.  Better still is to use profit not cost, and where you really want to go is value created over time because profitability can change dramatically, see this example:  Freemium Customer Conversion.

Media Not the Only “Attribution” Play

The discussion above is just the beginning of the attribution journey if your definition of “Marketing” is larger than “Advertising”.  All of the above is just about getting the customer to one action or purchase.  For some companies, that is all they care about.  But what happens once a person is a “customer”, however your company defines it?  How do you measure and optimize the downstream relationship?

There are Marketing Bands for these downstream stages of the relationship too, as well as when customers begin the dis-engagement process.  As with acquisition, different approaches are most effective for each stage of the dis-engagement process, and those should also each be measured on their ability to do a specific job.

As the LifeCycle of the customer plays out across time, as the funnel continues to narrow down to the final defection of the customer and resulting terminal LifeTime Value, there are optimal contact approaches for each stage or Marketing Band that you will need to discover for your business (click to enlarge chart).

It all starts at Satisfaction with the first goal achievement.   Those not satisfied defect (left side of chart).  Those who are Satisfied usually end up in a General Information communication cycle, e.g. weekly newsletter.  This is pretty much where current customer marketing practices end; people remain satisfied for some time then quietly move down through the At Risk and Dormant stages to Defection without Marketers noticing and are purged from the list (small black arrows).

However, those looking to maximize the profitability of the business  ask a further attribution question: what is the source / cause of At Risk and Dormant stage customers?  Is it the acquisition method?  Initial customer experience?  Product quality relative to expectation?  Service / experience after initial goal achievement?

What is learned from attribution analysis of At Risk and Dormant customers is then used to:

  1. Correct problems in the front end of the business that cause customers to defect in the first place
  2. Develop customized communications (Behaviorally Targeted / Last Chance) to drive customers back up into a state of  Satisfaction where they enter the Lifecycle anew, e.g. they start opening weekly emails again (blue arrows)

The right message, to the right person, at the right time.

For examples of  a measurement and action framework for driving increased business value from these “lower funnel” customers, see the Measuring Engagement Series.  Those not having the required customer analysis capabilities to attack this area could upgrade their digital analytics platform or use a tag-based tool optimized for this purpose like the LifeCycle Grid feature offered by Listrak.

Folks, the Marketing Funnel is not dead, for most it’s yet to be discovered.  What’s dead (or should be) is the way people currently think about the Marketing Funnel and measure the implications of it.

For more on the future of attribution and how you can leverage it, come to my eMetrics Boston presentation or grab me at the show, I’d be glad to wrestle the topic to the floor with you!

It’s become fashionable to declare the “Funnel Model” to be dead.
For example, here is a post worth reading on this topic by Rok Hrastnik.  There are some very good points in this post on why
trying to measure marketing value using a funnel model is really an insane idea.  Primarily, assuming you can track all possible
touch points and understand the value of each is not reality.
But what this really means is sequential attribution measurement approaches do not create insight into the value of each marketing
contact.   The funnel itself is not dead, what’s dead is the current model of measuring success.
Funnel thinking is based on a relatively reliable model of human behavior, AIDA.  This model from human psychology does not
specify tools, channels, or media.  It simply says that there is a path to purchase most humans follow.  That is:
A – Attention (Awareness): attract the attention of the customer
I – Interest: (Intent) raise customer interest, promote advantages and benefits
D – Desire: convince customers the product will satisfy their needs
A – Action: lead customers towards taking action / purchase
This AIDA funnel has not changed and it’s not dead.
People have to become Aware of a product or service to be interested in it.  They have to be Interested in order to Desire it, and
unless they Desire it, they won’t take Action to acquire it.
Example: I’m Aware of tons of products I would never buy.  There are lots of products I think are Interesting but I have no Desire
for.  There’s a short list of products I Desire but have not Acted on.  The list of products in my head worthy of purchase
consideration gets smaller and smaller at each stage of the AIDA model.  This is the funnel.
Said another way, it’s not the tools, channels, or media that are a funnel, it’s the way humans process buying decisions that is a
funnel.  And it’s pretty much always been impossible to measure marketing impact on this process in any kind of linear way at the
individual or even the segment level, because as Rok and others have pointed out, that impact is (usually) not linear.  Never has
been.
That is why media mix modeling was developed.  Since it’s really impossible to measure the incremental contribution of any one
marketing channel or message at the individual level without controlled testing, and controlled testing is often not possible in
media environments, you have to model the mix in the aggregate.
It’s never really been possible to create a step-wise marketing measurement funnel or the representative sequential attribution
model.  Lots of people wanted one, for sure, and of course vendors stepped in to provide one.  But these sequential attribution
models do not actually measure anything, they are simply lists of events.  Interesting stuff, and can be used to speculate or
imply, nice to know, great for the development of tests to prove incremental behavior and profit – but not measurement.  This is
what’s broken with a lot of people’s current thinking on the “funnel model”, and deserves to be dead.
.
In other words, what’s dead is the idea one can:
1. Accurately measure the path of people through Marketing exposures
2. Attribute value contributed by each exposure or funnel step simply by finding the step was involved in a path to purchase
3.  Use this information to control or optimize the sequence of exposures
Exposure does not automatically affect value; otherwise, surely we would see negative attribution, correct?  Have you ever been
negatively impacted by an ad or social exposure, like product reviews?  Of course you have.  At the same time, has anyone ever
seen one of these sequential attribution models give a negative value to certain campaign or media exposures?
Why is that?  How can that be?
The proper way to actually measure this kind of funnel idea is through media mix models, controlled testing, or both.  Here still,
you won’t be able to look at the effect of different exposures on *individuals* or specific sequences, but the effect of different
levels of media mix on groups or segments.  You cannot predict or control the sequence or level of media exposures, but what you
can do is try different levels and mixes of media, and correlate to sales or other actions.
The fact many orgs don’t have the tools or institutional stamina to do this kind of measurement properly should not result in
people declaring they have measurement of these effects when in fact all they have is a list of media exposures that is clearly an
incomplete and broken picture.
Hopefully, that approach is dead as well.
If folks can’t measure these funnel effects properly, they should just say so and ask for the resources to measure them correctly.
There’s no shame in saying you just do not have the right tools or enough resources.  If you have to provide metrics, just be
honest about what they don’t mean.  But please, don’t sit around and assign arbitrary values to exposures based on budgets or
whatever else.
A list of events is not a measurement, and you hurt the marketing measurement cause when you just make stuff up.  Trust me, senior
management knows you are full of crap.  And people wonder why analysts don’t get respect…
For those of you who want to proceed on attribution measurement but don’t have the resources to do it properly, there are
alternatives.  They’re not perfect, but at least it’s a real measurement that provides direction you can act on.
The easiest alternative is to focus on first click or action and measure value creation over time, since it’s the first action
that is generally most predictive of end value.  Why?  The facts surrounding the way a potential customer becomes aware of a
product or service and takes action to move down the funnel reliably predicts the experience they have on the rest of the journey
to final action.  You can’t control this journey, but you can measure the end result.
For example, compare the value of new customers 6 months after they became customers by source of first action.  The media, offer,
copy, process experience is reliably predictive of segment value at 6 months on a relative basis.  If certain campaigns generate
“high value customers” and others “low value customers” (however you define those ideas), they will continue to do so in the
future.
If you have the capability to get a bit more sophisticated than the above but are not ready for mix modeling or controlled
testing, here’s a framework for you.
Most experienced marketing people would agree different types of media are more efficient at addressing the various steps in the
AIDA path versus others.  I refer to this idea as “Marketing Bands” versus a “funnel”, it’s more like a stack of layers people
pass through rather than a specific path.  For each layer, there then to be unique media and messages that  are most effective at
moving people to the next layer and toward final action.
In other words, instead of looking at steps in a funnel and trying to attribute the contribution of each media step to the end
value of the journey, look at each AIDA step by itself and the success of a media *within* that step.  Why is this a better
approach that sequential attribution?  People can and will bounce around between the AIDA layers as they move through different
platforms, media, and messages, but that’s not a problem, it’s an opportunity in this framework.
For Awareness, secure Awareness-generating media and measure the effectiveness of those media at generating Awareness using
Awareness metrics.  Forget the rest of the funnel for these Awareness media, the next steps in the AIDA funnel are not the primary
success metric of Awareness media.
If Awareness is generated, Interest may follow.  Secure Interest-generating media and measure the effectiveness of those media at
generating Interest using Interest metrics.  Forget the rest of the funnel for these Interest media, the next steps in the AIDA
funnel are not the primary success metric of Interest media.
And so forth.  A visual of this idea is here:
http://www.jimnovo.com/images/i-marketing-funnel.jpg
(1993 HSN media choices in black type, 2008 media choices in red type)
Those interested in more on this idea check out the Marketing Bands series here:
http://blog.jimnovo.com/marketing-bands-series/
The financial results of such a Media / Marketing / Service effort, one that adds customer centricity as part of the marketing
mission, are here:
http://blog.jimnovo.com/2008/06/29/marketing-bands-numbers/
I’m sure some people will say, “But Jim, one piece of media can have effects in different AIDA layers.  For example, a campaign
designed to generate Awareness might also create some Interest, Desire, or Action.  How do you measure and account for these
spillover effects?”
The correct answer is a media mix model.  Do you have a media mix model?  Do you have the staff and corporate stamina to properly
execute the testing required to build one?  No?  Then I’m sorry, you’re just going to have to deal with an imperfect marketing
measurement system, and realize there will be beneficial effects you cannot measure.  If what you can measure properly is
successful, you can take comfort in knowing the actual results are probably even better.
Yes, the Marketing Bands framework is an imperfect approach, but without a mix model, it’s the best you’re going to get.  And it’s
a whole lot better then fabricating attribution out of thin air using a list of sequential exposures, a model that is broken in so
many obvious ways if you want to measure campaign effectiveness.
Summary
Sequential attribution provides us interesting information but it cannot properly measure the contribution of various campaigns to
an end marketing result.
The best way to so marketing attribution work is to run a media mix modeling project or for specific situations, controlled
testing.  However, there are significant challenges to doing this work and most companies will not be able to pull it off for
logistical or resource reasons.
The easiest alternative to media mix models and / or controlled testing is to figure out which campaigns / media / experiences
start the most valuable trips through the Marketing Bands – trips taken in whatever sequence the customer chooses.  Once you have
some learning from doing this, try optimizing these ideas by working each of the Marketing Bands as a distinct media impact model.
Finally, once you have been through these two approaches you will have plenty of data and success stories to ask for the resources
required to step up to media mix models and / or controlled testing – and a solid reputation with management that supports asking
for the resources to go for this end game.
However…that’s just the beginning of the journey if your definition of “marketing” is larger than “advertising”.
All of the above is just about getting the customer to first action or purchase.  For some companies, that is all they care about.
But what happens once a person is a “customer”, however your company defines it?  How do you keep a customer, measure and
optimize the downstream relationship?
If you looked at the graphics at the links above, you saw there are Marketing Bands for these downstream stages of the
relationship too, when the customer beings the dis-engagement process.  As with acquisition, there are different media /
approaches that are most effective for each stage of the dis-engagement process, and those should also be measured on their
ability to do a specific job.
As the LifeCycle of the customer plays out across time, as the funnel continues to narrow down to the final defection of the
customer and resulting terminal LifeTime Value, there are optimal contact approaches for each stage or Marketing Band that you
will need to discover for your business.
The right message, to the right person, at the right time.
Folks, the Marketing Funnel is not dead, for most it’s yet to be discovered.
What’s dead is the way you currently think about this Funnel and measure

Marketing to Focus on Customer. Analytics?

It’s been very popular among marketing types to talk about “the customer” but seek metrics for affirmation other than those based on or derived from the customer.  Web analysts have followed their lead, and provided Marketers plenty of awareness, engagement, and campaign metrics.  As I’ve said in the past, this is a huge disconnect.  Does it make sense (analytically) to have discussions about customer centricity,  customer experience, customer service, the social customer, etc.  and measure these effects at the impression or visit level?

Is someone who visits or purchases or comments one time really a customer, for the purposes of analyzing “centricity” ideas and concepts?  I think not.  Visit metrics simply don’t work for understanding these customer concepts, because by definition they unfold over time, not as single events.   Add in the fact most web activity is 1x in nature – even buyers – and you begin to realize that analyzing “traffic” yields very little in the way of “customer” insight.

From a Marketing perspective, hey, happy to have the 1x revenue, but these are interactions I’m not really excited about increasing spend on, knowing they will be a one-night stands.  This is especially true when you also know re-allocating some of the funds spent on the 90% 1x-ers to the other 10% could double company profits!

If you have followed my writings over the past 12 years, none of the above perspective is new.  What might be changing is this: more people in the online world are beginning to think the same way.

Continue reading

“Missing” Social Media Value

I have no doubt there is some value in social beyond what can be measured, as this has been the case for all marketing since it began ;)  The problem is this value is often situational, not too mention not properly measured using an incremental basis (as you point out).
For example,  to small local businesses who do no other form of advertising, there is a huge amount of relative value to using social media, versus no advertising at all.  Some advertising is much better than none, and since it’s free, the incremental value created by (properly) using social is huge.
On the other hand, I wonder why social analysis seems to forget that people have to be aware of you to “Like” you in the first place.  Further, it seems unlikely a person would “Like” a brand or product if they have not already experienced it, and are already a fan.  If this is not true, if people “Like” a company even thought they do not (paid to Like?), then the problems with social go way beyond analysis…
But if true, , the number of “Likes” doesn’t have as much to do with awareness as it does with size of customer base, and is much more aligned with tracking customer issues (retention, loyalty) than anything to do with awareness / acquisition.
Add the fact many companies are running lots of advertising designed to create awareness, and the incremental value of social as a “media” may be close to zero, or at least less than the cost to analyze the true value of it.
And this last, really, is the core of the issue.  It’s simply not possible to measure “all” the value created by any kind of marketing, and there are hugely diminishing returns as you try to capture the last bits.  I think it’s quite possible the optimism for “value beyond what can be measured” is less than the cost of measuring it *if* people keep looking in the awareness / acquisition field.
Folks who want to find this “missing” social value should start doing customer analysis, and look in the “retention / loyalty” area, where the whole idea of social is a natural, rather than a forced, fit.

Has to be There

I find it really interesting that whenever there is a discussion of measuring the value of social media, there’s such a bias towards believing there is value in social beyond what can be properly measured.  See the comments following this post by Avinash for a good example.  Speculation is fine, but the confidence being expressed that a new tool or method will uncover a treasure trove of social media value seems un-scientific (as in scientific method) at best.

I don’t doubt there is some value in social media beyond what can be measured, as this has been the case for all marketing since marketing measurement began.  These measurement problems are not new to social either:  Marketing value created is often situational, it depends on the business model and environment.  What works in one situation may not work in another.

For example:

To small local businesses who do no other form of advertising, there is a huge amount of relative value to using social media, versus no advertising at all.  Social advertising is much better than none, and since it’s free, the incremental value created by (properly) using social is huge.  It’s also really easy to measure the impact and true value, since the baseline control is “no advertising”.  Lift, or actual net marketing performance, can be pretty obvious in his case.

On the other hand, many companies are running lots of advertising designed to create awareness, and the incremental value of social as a “media” may be close to zero for these companies, or at least less than the cost to analyze the true value of it.  Possible explanation:  Social events such as “Likes” or comments are simply representations or affirmations of awareness already created by other media, so by themselves, create little value.  In other words, events such as Likes might track the value of other media spending, but may not create much additional marketing value.

Continue reading

Defining Behavioral Segments

The following is from the April 2011 Drilling Down Newsletter.  Got a question about Customer Measurement, Management, Valuation, Retention, Loyalty, Defection?  Just ask your question.  Also, feel free to leave a comment and I’ll reply.

Want to see the answers to previous questions?  Here’s the blog archive; the pre-blog newsletter archives are here.

Q: I purchased your book and have a few questions you can hopefully help me out with.

A: Thanks for that, and sure!

Q: We have 4 product lines and 2 of them are seasonal. i.e we have customers that year in year out purchase these items consistently but seasonally, for example, every spring and summer.  Then they are dormant for Fall and Winter.  Should I include these customers along with everyone else when doing an RFM segmentation?

A: Well, it kind of depends what you will using the RF(M) model for, what kinds of marketing programs will be activated by using the scores. If you know you have seasonal customers and their habit is to buy each year, AND you wish to aim retention or reactivation programs at them, I would be tempted to divide the customer base so that seasonal customers are their own segment.  Then run two RF(M)  models – one for the seasonals, and one for everyone else.

Q: If I include seasonal customers, and I run RFM say on a monthly basis, these seasonal customers will climb / fall drastically with time depending on the season, so it seems like it may complicate the scoring process.

A: Sure, and you could segment as I said above.  Or, you could run across a longer time frame, say across 2 – 3 years worth of data. This would “normalize” the two segments into one and take account of the seasonality in the scoring – perhaps be more representative of the business model.  However, the scores would become less sensitive due to the long time frame so the actions of customers less accurately predicted by the model.

Q: Can you provide me with some examples as to how segmentation is carried out?  Let’s say I being with RFM and all my customers are rated 5-5, 5-4, 4-5 etc.  What are the next steps, do we overlay with other characteristics like age, gender, etc?  Or are the 5-3 etc. our actual segments?

A: This goes back to what you want to use the RF(M) model for.  In the standard usage, each score will have roughly the same number of customers in it, those with higher scores will be more likely to respond to marketing and purchase, lower scores less likely.

Continue reading

Increase Profit Using Customer State

The following is from the March 2011 Drilling Down Newsletter.  Got a question about Customer Measurement, Management, Valuation, Retention, Loyalty, Defection?  Just ask your question.  Also, feel free to leave a comment and I’ll reply.

Want to see the answers to previous questions?  Here’s the blog archive; the pre-blog newsletter archives are here.

Q: We’ve been playing around with Recency / Frequency scoring in our customer email campaigns as described in your book.  To start, we’re targeting best customers who have stopped interacting with us.  I have just completed a piece of analysis that shows after one of these targeted emails:

1. Purchasers increased 22.9%
2. Transactions increased 69%
3. Revenue increased 71%

A: There you go!

Q: My concern is that what I am seeing is merely a seasonal effect – our revenue peaks in July and August.  So what I should have done is use a control group as you described in the book – which is what I am doing for the October Email.

A: Yep, that’s exactly what control groups are for – to strain out the noise of seasonality, other promotions, etc.  But don’t beat yourself up over it, nothing wrong with poking around and trying to figure out where the levers are first.

Q: Two questions:

1.  What statistical test do I use to demonstrate that the observed changes are not down to chance

2.  How big should my control group be – typically our cohort is 500-800 individuals

A: Good questions…

Continue reading

All Talk, No #Measure

Hypocrisy in Web Analytics?

Before every eMetrics (I’ll be in San Fran teaching Basecamp, at the Gala, etc.), I try to ask myself, what is the most critical issue facing the web analyst community right now?  Then, at the show, I ask everyone I run into what they think about this issue.

There’s lots of issues to choose from.  Career path I think is a big area of discussion, given the mergers in the space and trend towards outsourcing.  Then there’s the “we don’t get no respect” thing; senior management doesn’t seem to listen / understand / act on the information provided.  And one of my favorites from the past is still out there, data torture - people being pressured to manipulate data to reach a predetermined analytical outcome.

But seems to me, more important at this juncture is trying to resolve why there is so much written about the importance of “the customer” but very little measurement at the customer level.  Think about it.  Customer experience, customer centricity, the entire social thing, it’s all about customers.

But when folks wants to trot out “proof” that this or that approach is the road to the promised land, they analyze impressions, visits, clicks, etc.  Visitor-level stuff.  Does that seem like the correct approach to you?  Seems to me, if you want to provide knowledge about customers, you should measure customers.

Continue reading

But What is an Impression Worth?

Seems like coming up with a value for social media has become a cottage industry, for example, $3.60 Facebook Fan Valuation Is Just the Tip of the Iceberg.  These values are often derived from what is paid for online media.  So you have to ask, if someone is basing the value of a Facebook fan on the value of impressions generated, what is the real value of those impressions?  Because unless this is known, the whole framework is faulty.

Just because you pay $5 / CPM for impressions, does not mean they are worth $5 / CPM, does it?  Do people really still have that kind of mentality?  Is the price of the media equivalent to its value?

For example, I’m sure you have heard of multi-million dollar campaigns that generate very little lift in sales.  Happens frequently in fast food, for example.  What is the value of that media?  Is it the millions paid?

What really blows my mind about this approach is it’s so offline, so old school PR. Do the folks who put forth these kinds of models believe nothing has changed in 50 years?  What happened to the whole rap of online being “different”, that you can’t measure it like offline, blah blah.

Except when it’s convenient to do so?

If you want to know the value of a Facebook fan, why not measure the value of a Facebook fan?  Because it’s hard, and would require organizational discipline?  Too bad.   Substituting the kind of models used in the example above for actually measuring the value of a Facebook fan is misleading at the very best.

Continue reading

Optimizing for Customer Value

The following is from the February 2011 Drilling Down Newsletter.  Got a question about Customer Measurement, Management, Valuation, Retention, Loyalty, Defection?  Just ask your question.  Also, feel free to leave a comment and I’ll reply.

Want to see the answers to previous questions?  Here’s the blog archive; the pre-blog newsletter archives are here.

Q: Thank you for creating this useful website!

A: You’re welcome!

Q: When figuring out retention rate for an annual or a 8 months life time cycle period, how do I pick the starting period?  Do I look at their first orders on a date?  Or I pick a time frame such as one month?

A: It depends on:

1. What kind of “retention” you are talking about, the definition, which is probably impacted by the audience for the data

2.  What you will do with the retention data, what kind of decisions will be made and actions be taken because of the data

You should always ask these questions above  when someone requests “retention data” – or any other kind of analysis, for that matter!  For example, there probably is a huge difference in what you would provide to the Board of Directors for an annual benchmark and what you would provide to Marketing people for executing campaigns.

Continue reading

When Does a Visitor Need a Coupon?

The following is from the November 2010 Drilling Down Newsletter.  Got a question about Customer Measurement, Management, Valuation, Retention, Loyalty, Defection?  Just ask your question.  Also, feel free to leave a comment and I’ll reply.

Want to see the answers to previous questions?  Here’s the blog archive; the pre-blog newsletter archives are here.

Q: First off, I very much appreciate you sharing all this wonderful content on your blog and conferences such as eMetrics.

A: Thanks for that!

Q: My question is a simple one, but I think the answer may be hard: When does a visitor “need” a coupon?  *Need* defined as: visitor would not have placed an order unless presented with the coupon.

A: Hmmm…methinks we’re going to have to define a few concepts and be clear on the goals to make sure we are nailing this down… visitor versus customer, sales versus profit, etc.  In other words, answer is not hard, but could be complex without defining context.

Q: It’s still a mystery to me why so many retailers seem more than willing to hand over all their margins to Groupon or give coupons to basically all visitors.  I am curious whether you would approach this question using  observational data (eg web analytics) or experiments (eg AB testing), or both.

A: Right – is a mystery to me too!

There are certain situations where this approach might be appropriate, but the problem with much web “marketing” (which often is really just advertising without much thought about marketing) is often there is success in a narrow or special situation.  Then the pundits jump on and say “if you’re not doing this you are stupid”, regardless of the business situation and / or without recognizing the special circumstances that are driving success.  This is all the real Marketing stuff people leave out; understanding why it works, under what circumstances, for which segments, involving which products.

Continue reading

Freemium Customer Conversion

The following is from the October 2010 Drilling Down Newsletter.  Got a question about Customer Measurement, Management, Valuation, Retention, Loyalty, Defection?  Just ask your question.  Also, feel free to leave a comment and I’ll reply.

Want to see the answers to previous questions?  Here’s the blog archive; the pre-blog newsletter archives are here.

Q: I was wondering if you’ve done any work with, or given thought to, companies who have a cloud based Freemium business model?

Should they be tracking usage (or anything) at the free level?  Should they be tracking usage at the paid level?  I’m sure defection rates are a big problem, but I’m wondering how many focus on engagement thru mass marketing versus trying to keep what they’ve got, or influence the free users to make the leap to paid.  Any thoughts on this?  Maybe you could do a blog post on it.  It seems like a good fit with your brand of analysis but I’m just starting to think it through…

A: I just finished an analysis that’s a good example of this problem.  Behavior during the Freemium period can predict who is highly likely to become a paying customer, who will need marketing efforts like additional sampling / package discounts, and who will not become a customer no matter what you do.

Continue reading