Not all of these programs are Marketing, some are Service, and some could be considered “Operations”.  But they all touch the customer, so we were thinking changes in customer value might be a common way to measure and compare the success of these programs.

A: Absolutely!  I just answered a question very much like this the other day, it’s great that people are becoming interested in customer value as the cross-enterprise common denominator for understanding success in any customer program!

If I am the CEO, I control dollars I can invest.  How do I decide where budget is best invested if every silo uses different metrics to prove success?  And even worse, different metrics for success within the same silo?

By establishing changes in customer value as the platform for all customer-related programs to be measured against, everyone is on an equal footing and can “fight” fairly for their share of the budget (or testing?) pie.  By using controlled testing, customers can be exposed to different treatments and lift in value can be compared on an apples to apples basis – even if you are comparing the effect of a Marketing Campaign to changes in the Service Center.

But are you sure you want to use LifeTime Value for this application?

Q: From what you stated on your website, I will not be able to develop a LifeTime Value model unless I understand the customer Lifecycle.  The customer lifecycle is something that I could get a good understanding from using doing a RFM analysis.

My question is, once I complete the RFM analysis, what would be my next steps in developing a customer LifeTime Value model?   At this point in time, the hardest thing that I am trying to wrap my head around are the variables to include in the model.  I visited Arthur Middleton Hughes’ website:

http://www.dbmarketing.com

and he suggests the following variables (download spreadsheet, if interested):

http://www.dbmarketing.com/special_ltv.htm

Jim, could I simply use those variables going forward to calculate the LifeTime Value of a customer at my company?  I would appreciate any assistance you may be able to provide to me on this matter.  Thanks.

A: Well, that’s a big tangle of related issues!    Let’s unpack first, then answer the question.  First, the relationships between these ideas:

Lifetime Value versus Lifecycle – LTV is a number, LifeCycle is a trend over time that contains trigger events.  You don’t need the LifeCycle to develop (calculate) LTV, you need the LifeCycle to most efficiently and profitably act on and manage LTV issues.

RFM versus Lifecycle – RFM is a tactical model that is a “snapshot” of customer state at a point in time, the customer’s likelihood to respond.  Frequently used names for these customer states include active, lapsing, lapsed, defected.   Lifecycle is the “movie” one might put together from these snapshots of RFM states; the migration from one customer state to the next are the Lifecycle trigger points.

Now, let’s make sure we understand each one of the ideas:

LifeTime Value

Strictly speaking, LTV is not a very flexible concept and is best used for determining how much you can spend to acquire a customer and still make a profit.  This is the equation that Mr. Hughes has provided, a man by the way that I have a lot of respect for.  His model is quite detailed and useful for the purpose of finding break-even cost to acquire a customer.

To use Arthur’s LTV model, you have to find historical values and plug them in.  You could assume nothing will change and the LTV of certain segments of past customers will be the same; this is great for “benchmarking”, for example.  However, this approach is not measuring LTV, it’s predicting LTV based on historical data.  This is fine, and a valid method for certain types of analysis.

But, the premise of your question is you will be testing, and testing implies something new will occur.  So while you could use LTV to estimate results, you’d have to wait quite a while to prove the results one way or another.  LTV is really “forensic” in this way – you won’t know the final answer until the customers defect.

You could certainly go back 2 – 5 years after the tests, and prove one group had higher LTV than another, but that’s not typically a very useful approach when doing testing.

RFM (Recency, Frequency, Monetary)

RFM is a predictive model that takes a “snapshot” of the customer base and gives you a score for each customer, a prediction of likelihood to respond relative to all customers.

By itself, RFM doesn’t tell you if you are making money or not.  It is used to classify the “state” of customers at a point in time, usually for targeting purposes – are they active, lapsing, lapsed, defected?  In other words, it’s a customer segmentation tool.

For example, RFM could be used to choose your test and control groups for a campaign using Lift measurement – you would want test and control to have the same range and balance of scores.  In fact, one of the tragic campaign measurement mistakes people often make is not taking into account the likelihood to respond when selecting test and control groups, resulting in biased test results.

Customer LifeCycles

One of the great features of RFM is the idea of “ranking” customers relative to each other; this gives allocation of budget and success measurement a standard to follow.  A single  customer can have many different scores over the course of their LifeTime, with the likelihood to respond the score at a specific time.  In fact, if you looked at RFM scores over time for a single customer, you would have a clear understanding of the LifeCycle of a customer – the most powerful segmentation available in terms of message and offer targeting.

The problem with looking at RFM scores over time is complexity; the beauty of individual customer scores at a single point in time becomes unbearable when you are talking 125 different scores on 50,000 customers over 6 months.  That’s the internal or analytical problem.  Externally, this kind of information is extremely gnarly to present and explain to senior managers, it’s presentation hell.

The way I solve this problem is with a tool I call LifeCycle Grids.  The Grids takes the same fundamental drivers used in the RFM model and instead of ranking, uses thresholds or “hurdles” to classify customer states.  This creates a standardized customer LifeCycle “dashboard” so comparisons of customer value between different segments can be made more easily.  It works for both short and long term observations and is easy to represent either numerically or graphically.  And because it uses finite thresholds for activity rather than ranking, the same calculations that create the dashboard can be used to actually drive or trigger actions.

So the dashboard is actually the controller as well.  This is extremely beneficial in terms of linking presentations, plans, and results. People can literally point to a segment on the LifeCycle framework and say, “Let’s deliver message X to each person from segment Y who enters this cell” and see the results right where they pointed when the dashboard is updated.

Once you test some ideas and find out which approach generates incremental profits for a cell in the Grid, you can automate delivery of the program as customers enter that cell of the Grid.  This is the classic “sense & respond” approach to marketing communication – right message, right person, right time.

The LifeCycle Grids are demonstrated in a lot of detail for different applications in the series here, but probably of most interest to you as it relates to customer analysis, see here.

And now, to answer your question:

Which approach above, if any of these, would be best for standardizing measurement of ROI in widely diverse customer programs?

LTV would be appropriate if what you want to know is breakeven cost to acquire.  Since we are talking about customer programs, I doubt that’s what you want to use.  Plus, if you want a hard number rather than a prediction, you could be waiting a long time for the answer.

RFM is a “snapshot” model and so not really suited to long-term studies of customer value.

Customer Lifecycle models are more likely to be involved in the execution of a program, not the success measurement.  LifeCycle tracking could be (and often is) used to predict the financial success of campaigns before they have run their course, but you’re only predicting success, not delivering numbers into an ROI model the CFO would accept as “fact”.

Answer: None of the above.

What you need is an approach designed for the task, which in this case, is:

Lift Measurement or Near-Term Value

Lift is a measure of the performance of a test group of customers compared with a control group of similar customers who are not exposed to the test.  You can read more about control groups here.  In the analysis of value contributed by each group, many of the same values from Arthur’s LTV model are used – product margin, costs of program, fulfillment costs, payment parameters, etc.  However, if you are talking about a program to existing customers, cost to acquire is probably not relevant, though you might use source (campaign) to segment your test approach.

Lift is typically measured at intervals, say every 30 or 60 days, to see how test versus control populations are tracking, and can continue after the test is over to pick up residual value created in the customer.  However, this is not a Lifetime Value measurement, Lift models measure incremental contribution to LTV created by the Marketing, Service, or Operations program execution.

This means if you get lift from program test versus control, when you go back 2 – 5 years later and measure true rather than predicted LTV – after the customer has defected – you should in fact see the LTV in the test group higher than in the control group, barring any radical downstream difference in customer experience between test and control.  In this way, Lift models are actually predictive of changes in LTV.  That’s why the output of Lift models is sometimes referred to as the measurement of “Near-Term Value” and used much more often than the forensic approach of waiting for customers to defect.

Summary

All the above are core concepts in customer value measurement and management.

LTV is a measurement of net financial value contributed by a customer, and Lift measures  are like a “time slice” of the overall LTV curve.

LifeCycles are a management framework for programs designed to affect LTV, and models using Recency, Frequency, and Monetary are used to look at a “time slice” of the LifeCycle.

LTV can generally be increased in two ways: by creating more value during the existing LifeCycle, or by extending the LifeCycle.  Marketing (including Product) is typically used when doing the first, Service and Operations – customer experience and satisfaction – are largely what affects the second.

So it is completely appropriate to establish a unified approach to the measurement of customer programs intended to increase the value of a customer across all these disciplines, in order to ensure the allocation of  scarce resources to highest and best use.

A great question, and for a great cause!

Jim

Update:

Listrak asked me to do a podcast with them on these and related topics, check it out (MP3 link) here, or see list of all their Email Marketing Today podcasts here (I’m on Episode 42).

Have a question on Customer Valuation, Retention, Loyalty, or Defection? Go ahead and send it to me here. If on the topic above, you can leave a comment on the post:

LTV, RFM, LifeCycles – the Framework

So here, I will attempt to answer other kinds of questions I think people might have based on my discussions with WAA members.

Update: The WAA has answered many Certification questions here.

However, I’m going to approach this topic a bit differently than most of the published documentation – from a Product / Marketing perspective, rather than an Educational / WAA POV.  I can do this because (if you don’t know) I have worn all the hats on this project – developer, marketer, WAA project owner – and I think it might be helpful to tell the business story of the WAA Certification, from the bottom up.

And if you have other questions, feel free to leave them in Comments and I will do my best to answer them!

Where did the idea for Certification come from?

The WAA is a member-driven organization; we listen to the membership and try to accomplish what they would like us to accomplish.  We heard from hiring folks and managers that ”web analysts today know a lot of the buzz words and can follow instructions as far as reporting goes, but what we’d be willing to pay a premium for is web analysts who discover things on their own, who add value in areas we don’t already know about”.

So that’s where WAA Certification came from.  It addresses a specific need identified by members, what came to be known internally as the “Book Smart versus Sherlock Holmes” problem.  Sure, you can read a ton of books or blogs and be a  good web analyst by following best practices.  But so can a lot of other people.  What you need to pass the Certification Test is different; you have to be able to turn data into insight and recommend a best action given the scenario presented.

This is Why the WAA’s Educational efforts lack “tool focus”?

Sure.  And of course, the tool vendors already own that focus, and by definition they have the resources to be much better at tool education / certification than the WAA, so why would the WAA want to compete with the tool vendors in the same space?

Better to add value on the business side, where there is demand we can fill and a lack of trusted resources.  And if you think about it, this approach simply expands the overall WA opportunity.  People who want to become experts on the tool side have a path (through the vendors), and people who want to become experts on the analysis / business side also have a path through the WAA.  And if you want to be a Universal Web Analytics Soldier, I guess you could do both!

Does that mean I can pass the Test with No Tool Knowledge?

Not at all.  The threshold we set is you need to be able to communicate effectively with tool experts to pass the test.  That means you will need to know the basics of how the web works, how the tools accomplish their mission, and know what all the web analytics terms mean.  Example: To pass the Test, you don’t need to know how to write a tag, but you do need to know when a  custom tag  is required and how to communicate your need effectively.

So Marketing people can Pass the Certification Test?  eCommerce Managers?  Usability people?  Media Buyers?  Etc.?

Absolutely, if they are good at transforming the data generated by web analytics tools into business insight AND have broad knowledge across the entire scope of web analytics.

Who Created the Certification Test and How?

About 50 WAA members from all over the world volunteered to take on the task.  We created questions, tested them across different audiences, gathered feedback, rewrote the questions based on the feedback, tested the questions again.  You know, the continuous improvement thing?

If you want to participate in the ongoing process of creating the Certification Exam, there is more info here.  Please note you have to be a member of the WAA to be on any WAA Committee.

Where did the Requirements to take the Test Come From?

From the 4 Test the Test sessions we held at various eMetrics events, where we asked people to volunteer to take the Test.  We looked at the backgrounds of  people with high scores versus people with low scores and established the  benchmarks.  People with higher than average scores had these characteristics:

Years of Web Analytics Experience:  5.4
Interprets reports / suggests actions to be taken: 100% of population
Training / Courses in web analytics:  100% of population
Education post High School: 4.8 Years

People with lower than average scores had these characteristics:

Years of Web Analytics Experience:  2.3
Interprets reports / suggests actions to be taken: 50% of population
Training / Courses in web analytics:  63% of population
Education post High School: 3.6 Years

But inside these averages (segmentation!), it gets much more interesting.  Turns out the less experience you have, the more formal education / training helps you get a higher score.  Education could be college / advanced degrees, vendor training, or classes in web analytics / e-commerce.  Logical, and expected.

Not so intuitive was this on the mix of education and experience: when you have a lot of one and little of the other, you tended to get a lower score.  For example, both Ph.D’s with low years experience and people with 10 years experience but lacking education / training tended to get lower scores.  Likewise, people who indicated they “read blogs and books” as the only source of education did not tend to have high scores unless they had a lot of direct web analytics experience.   So somewhere in the middle there is a “magic mix” of experience and education that results in higher scores.

Interestingly, the single most reliable predictor of a higher score on the test was whether or not in the current job the person regularly suggests actions to be taken based on the analysis.  This data point is more subjective than years of education or experience so we did not include it as a requirement to take the Test, but it’s worth mentioning since it aligns closely with the purpose of the test.

In the end, it’s tough to predict tangible business analysis skills based on just education or experience alone, and this is why the Certification Test should be an important tool for people hiring web analysts.

I’ve heard the Test is Difficult to Pass; can you Explain Why?

In short, because we are a young industry and people tend to have narrow experience relative to the scope of the topic.

You can be an expert in e-mail and Display analytics and still not pass the test because you don’t know enough yet about PPC analysis or Optimizing Web Sites.  You don’t have to be an expert at everything to pass the Test, but you do need to have some knowledge across the entire scope of web analytics to get a high score.  See the Knowledge Required for Certification document for an overview of topics.

That said, I’m sure many of you have been faced before with challenges you did not understand or have any experience with – and then you figured out how to produce insight.  That brainset is precisely what the WAA is testing for.  So if you can take what you know from e-mail analysis and use it to figure out a question about PPC analysis, you could answer the PPC question correctly.  Do that enough times across the different knowledge areas and you could pass the Test, because you essentially demonstrated the ability to think analytically – the objective of the Test.

In opposition to that scenario, blindly following best practices in any knowledge area without recognition of the changes in approach a particular business situation or model might require means you probably will not pass the Test; you will need the capacity to modify your thinking based on the business goals presented.  Example: the correct answer for the publishing model may not be the correct answer for the commerce model.

How Do I Decide if I Should Take the Test?

Honestly, I personally think the Certification has much more value to people who are in the earlier stages of their web analytics  career.   Let’s say you have the same training and read the same books as a lot of other folks.  And you are trying to establish yourself as a person who can create business value but don’t have the resume to back that position up quite yet.  Passing the Certification Test could give you the edge you need to make things happen faster for you.

Conversely, if you have an awesome resume of accomplishments and references for those deeds, then why would you need the additional “proof” the Certification Test provides?  Plus, experienced people often specialize to distinguish themselves from the crowd, and a Test across the universe of Web Analytics would not be particularly relevant.

So I’d expect the majority of people taking the Certification Test to be say 3 – 4 years into their WA careers, or perhaps  earlier if they have been focused on WA and exposed to the right training or experience environments when doing the actual work.

The above is from the perspective of an individual.  However, an agency, consultancy, or service provider might decide having their analysts Certified (including senior people) creates a competitive advantage in their particular space.  Companies looking outside for analytics help may feel more comfortable hiring a resource with WAA Certified talent on staff.

Are there any Questions?

Feel free to ask about anything,  and please see the WAA FAQ for questions on execution details.

Have a question on Customer Valuation, Retention, Loyalty, or Defection? Go ahead and send it to me here. If on the topic above, you can leave a comment on the post:

Inside WAA Certification: Any Questions?

You may recall I wrote last year about the explicit or implicit pressure put on Analysts to “torture the data” into analysis with a favorable outcome.  In a piece called Analyze, Not Justify, I described how by my count, about 50% or so of the analysts in a large conference room admitted to receiving this kind of pressure at one time or another.

Since then, I have been on somewhat of a personal mission to try to unearth more about this situation.  And it seems like the problem is getting worse, not better.

I have a theory about why this situation might be worsening.

Companies that were early to adopt web analytics were likely to already have a proper analytical culture.  You can’t put pressure on an analyst to torture data  in a company with this kind of culture – the analyst simply will not sit still for it.  The incident will be reported to senior management, and the source of “pressure” fired.  That’s all there is to it.

However, what we could be seeing now is this: as #measure adoption expands, we find the tools in more companies lacking a proper analytical culture, so the incidents of pressure to torture begin to expand.  And not just pressure to torture, but pressure to conceal, as I heard from several web analysts recently.

One bright young analyst went “beyond the call of duty” on his analytical project.  The analyst gathered relevant data not just from the WA tool, but from Finance, Customer Service – all around the company.  The report painted a detailed picture of cost to acquire customers through various methods and campaigns, and was presented to the head of Marketing – also the analyst’s boss.

The analyst was told under no circumstances was this report to ever be produced again.  Further, the analyst was told to destroy any “evidence” this project / report ever existed.  And finally, the analyst would now be required to send all analysis through the boss first before anybody else sees it.

That’s shameful behavior for an exec.  And apparently, this kind of thing is happening more and more often.  I’ve heard plenty of “if we want your opinion, we’ll ask for it” stories, but this is the first time I’ve heard so many stories about concealing results.

Here’s a scary thought: what if the stories about web analytics not driving business value are primarily concealment stories?   What if the tool / analysts actually did provide value, which was then hidden from Senior Management?

My concern about this issue is wider than screwed up company culture and management.  What I’m more concerned about is screwed up people, analysts who may come to think this kind of behavior is normal and just part of being an analyst.

This matters because as this new generation of analysts moves to other companies and throughout the ecosystem, these pressure to torture situations could become “accepted” and even spread as “part of the game”.

It is never, ever OK to manipulate or hide the results of an analysis.  It’s not part of the job.  The role of an analyst is to analyze, not justify or conceal bad news.

Now, I realize some folks are thinking, “Yea, that’s great Jim, I’ll just get myself fired by being an analytical hero”.

I’m not saying you should respond to data torture pressure by falling on your analytical sword.  What I am saying is you – and management – need to know this kind of pressure from a superior is shameful, not a “normal” part of being an analyst.  And as soon as you can, you should get a job somewhere people respect your professional opinions.  Don’t have to agree; but must respect.

Like the company you work for?  Ask a buddy in Finance if they could use a web analyst.  Pretty sure Finance would be interested in fully-loaded cost to acquire new customers by source!

What really troubles me about this situation is it’s rarely ever talked about, so could be worse than people might think.  At the very least, Senior Management should know about the potential for this to happen and lay down some rules.  Perhaps even seek some cultural guidance on this topic (here’s a start – Fear of Analytics).

So, I want to put this message out there, perhaps create a resource for people who are looking for information on this topic.  It would be great to have examples so managers can understand and be on the lookout for these situations.  Plus, I’m sure there are some terrific stories out there about either giving in to the torture pressure or resisting it!

What about you?  Were you ever pressured to torture the data?  What happened?  Did you comply?  How did things come out?  Tell us with a Comment.  Feel free to post anonymously, leave out company names.

Have a question on Customer Valuation, Retention, Loyalty, or Defection? Go ahead and send it to me here. If on the topic above, you can leave a comment on the post:

Tortured Data – and Analysts

Want to see the answers to previous questions?  Here’s the blog archive; the pre-blog newsletter archives are here.

Q: Thank you for your recent article about Control Groups.  Our organization launched an online distance learning program this past August, and I’ve just completed some student behavior analysis for this past semester.

Using weekly RF-Scores based on Recently and Frequently they’ve logged in to courses within the previous three weeks, I’m able to assess their “Risk Level”– how likely they are to stop using the program.  We had a percentage who discontinued the program, but in retrospect, their login behavior and changes in their login behavior gave strong indication they were having trouble before they completely stopped using it.

A: Fantastic!  I have spoken with numerous online educators about this application of Recency – Frequency modeling, as well online research subscriptions, a similar behavioral model.  All reported great results predicting student / subscriber defection rates.

Q: I’m preparing to propose a program for the upcoming semester where we contact students by email and / or phone when their login behavior gives indication that they’re having trouble.  My hope is that by proactively contacting these students, we can resolve issues or provide assistance before things escalate to the point they defect completely.

A: Absolutely, the yield (% students / revenue retained) on a project like this should be excellent.  Plus, you will end up learning a lot about “why”, which will lead to better executions of the “potential dropout” program the more you test it.

Q: However, in light of your newsletter, I realized that we should probably have a control group with whom we do NOTHING (just as we did this past semester) in order to prove the effectiveness (or not) of the program.

A: Correct.  Otherwise, you won’t be able to make a valid claim to the “saved students”. People can always argue a variety of other factors were in play – seasonality, topic, course sequence, etc.

Q: Since the actual number of students is confidential, can you please tell me what percentage you would use for a control group if we had 400, 800, 1200, 1600, 2000, 3500, or 5000 students?  You mentioned 10% in your newsletter, but the population you were referring to exceeded millions.

A: Well, there are online calculators you can use confidentially, example right here.

If you don’t understand the variables they are asking for, explanations at bottom of page, though this is very simple – what is confidence level and interval plus population size.

Q: Our population is MUCH smaller, and each customer is therefore even more critical.  I don’t want to recommend an unnecessarily large control group that would prevent us from retaining future students when we could see they were having trouble.

I suspect that our defection rates will be lower 2nd semester than 1st since students should be beyond the “learning curve,” so I don’t think we can justly say that the program alone is the reason for lower defection rates if we don’t use a control group.

A: Yes, well, this desire to “get as much test as we can” was the main point discussed in the newsletter.  And that’s the challenge with very small populations – to hit statistical confidence levels at say population = 500, you need over 300 or so in control.

Not so great.

So we go back to the question of company culture and how intuitively confident people will be with the results.  Do they in fact need true statistical significance for a program like this?

There is a way around the significance issue – repetition. The stats part of this is all about the “likelihood you get the same results again” – real important for drug testing, not so much for 500 folks in a marketing program.

The question you need to ask: do you really need “prediction”?  Or does prediction just make the whole test more complex and expensive than it’s worth?  What if you repeated the test a couple of times and got roughly the same results, is that “proof”?

Here is what I might do.  I would ask whoever needs to believe in the results of this test a question like this:

“Let’s say we took a random 20% sample of the students and excluded them from the marketing.  We apply the marketing to the other 80% and their retention rate is 15% higher than the 20% who had no marketing. We do this test 2 more times and the retention rate of students in the test is 13% and 17% higher than the students in the 20% who do not receive the marketing.  Would you at that point believe that without question, the marketing drives at least a 13% improvement in retention among students?”

Do you see where I’m headed with this?  The more times you repeat the test, the more confident you will be in the results – regardless of sample sizes and statistical mumbo jumbo. At some point, the reality of the differences between test and control performance has to be accepted.  It may help to define up front how many repetitions the “boss” needs.

There are two clues to help you evaluate the validity of your results / how many times you need to repeat the test to be “confident”.

One clue is the variability of the results – the more inconsistent the results are, the more likely the data is “noisy” and the more times you need to repeat the test to be confident.

If the spreads between test and control for the first 3 tests are 20%, 5%, and 10%, then you’ll need more repetitions of the test to get a good feeling for the actual impact.  If the results tend to cluster as in the example above (15%, 13%, 17%) then you can be more confident earlier in the test series the actual impact is somewhere around 15%.

The other clue is in the “spread” between test and control.  If the spread is consistently  ”wide”, say +10% (or more), this provides additional confidence a positive impact is being made.  The result over a series of tests may not actually be +10% (confirm by repeating the test), but it’s more likely to be positive.  If you consistently get a spread more like 1% or 2%, it’s more likely the actual result could be zero or negative and you need to keep repeating the test to gain confidence you have a positive result.

In the end, you may not want or be able to repeat the test enough times to know with statistical confidence what the result is.  But if the spread between test and control is wide and consistent, and the cost relative to the benefit is small, then does it really matter if there is statistical confidence?

For example, if you can make the statement you’re confident the program generates at least $10 in profit for each $1 invested, does it really matter if the statistically confident  number is $11 or $12 profit for $1 in cost?  We’re doing Marketing here, not drug testing.  There is an opportunity cost (profit left on the table) to not rolling out a program based on a test with results like this; rather than repeat the test to death just to be more confident I’d roll it out and continue to monitor the results.

One more tip, on this idea of sequencing / semesters / experience with the program.

There is no doubt in my mind that 2nd semester students would have what is called a “survivor bias” and be less likely to drop out; you will get the best performance in a program like this with 1st semester students.  So if at all possible, run the test / control on only 1st semester students , or segment by semester.

But, just because you run it on only 1st semester students does not mean you don’t have an effect in 2nd semester.  Continue to follow test and control into 2nd, 3rd, 4th semesters and you may see the dropout rate of the original 1st semester group continue to widen versus control.

This is not only great for the profitability of the initial 1st semester program but also provides you the baseline you have to beat (control) for those 2nd, 3rd, 4th semesters.  When you decide to see if you can have an additional effect by intervening in those periods, you’ll have 2 groups: those affected by Marketing in the 1st semester, and those new to any Marketing intervention.

My guess: a 1st semester intervention will have tremendous impact, both then and throughout the 4th.  The impact of intervention at each subsequent semester will diminish compared with acting in 1st semester, as will the “tail” value created over the student life, since the number of months left in the student life is shrinking each semester.

Hope that helps!

Jim

Have a question on Customer Valuation, Retention, Loyalty, or Defection? Go ahead and send it to me here. If on the topic above, you can leave a comment on the post:

Control Groups in Small Populations

Want to see the answers to previous questions?  Here’s the blog archive; the pre-blog newsletter archives are here.

Q:  I am hoping you can help answer a question for our team.  By way of introduction, I am the CEO of XXXX.  We are a specialty retailer / restaurant of gourmet pizza, salads and sandwiches.  We would like to know  restaurant industry averages (pizza industry if possible) for customer retention – What percentage of customers that have ordered once from a particular restaurant order from them a second time?  I am hoping with your years of expertise and harnessing data you may be able to assist us with this question.  Look forward to hearing from you.

A:  Unfortunately, in those said years of experience, I have found little hard information on customer retention rates in QSR and restaurants in general (if anyone has data, please leave in Comments).  It’s just the nature of the business that little hard data, if collected, is stored in such a way that one can aggregate at the customer level.  The high percentage of cash transactions doesn’t help matters much; there’s a lot of data missing.

Over the years, sometimes you see data leak out for tests of loyalty programs, and of course clients sometimes have anecdotal or survey data, but this is not much help in getting to a “true” retention rate.  More often than not you discover serious biases in the way the data was collected so at best, you have a biased view of a narrow segment.  Often what you get is a notion of retention among best customers, or customers willing to sign up for a loyalty card, but not all customers.  And the large “middle” group of customers is where all the Marketing leverage is.

What to do about this predicament?  

There are really two issues in your question; the idea of using industry benchmarks when analyzing customer performance, and the measurement of retention in restaurants.

As far as industry benchmarking, two things:

1.  Annual reports for publicly traded eateries may be of help.  Customer loyalty info may be disclosed in these documents or conference calls with Wall Street.  Still, it will probably be of the quality referenced above – narrow in scope or behaviorally biased.

Sometimes you can put snippets of different conversations into an equation that allows you to guess at repeat purchase rate; hospitality analysts often want to understand repeat behavior and do this kind of forecasting.

2.  Ignore the industry benchmarks.  If you have the capability to track repeat rates, simply establish what they are now and use them as internal benchmarks to not fall below or create programs to improve against them.  

Frankly, I tend to discourage using “industry benchmarks” because the kinds of businesses that can really leverage repeat behavior and retention (customer-centric model) are usually *different* from the industry, so using a benchmark (say, from Domino’s) is probably low-balling your potential.  

Not that Domino’s is a “bad” operation, mind you, but they are what they are, they tend to be more on the operational excellence side of the game than customer intimacy (that’s what we called the customer-centric / social approach back in the early 90’s). 

Product leadership, the 3rd value discipline, is pretty much table stakes for anyone in the restaurant biz, and I assume from your business description you just might consider this a primary focus which you then leverage to create power in the intimacy area.  This is essentially the Apple Strategic model.  If the product is not great, the love will not come.

My point is this: without understanding the value discipline and Strategy of a competitor, you can’t know if any benchmark is something you want to compare to, because the business may have a completely different focus than yours.  Worse, using industry averages simply hides any real information you might gain that is actionable for your business.

For example, even though Walmart and Nieman Marcus are in the same business, I don’t think anyone would say they have the same Marketing Strategy or core value proposition.  Walmart is of course the poster child for operational excellence with the end result being value pricing, which flows to the advertising content.  There’s nothing “wrong” with this approach, it simply is what it is, and customer intimacy / relational / social marketing simply doesn’t really fit here.  You certainly can try to be as intimate as possible; but it must be done within the constraints of the model and not reduce operational excellence.  Importantly, this is a “mass” concept, so Push media is the most effective.

Sam’s Club is an example of how one might accomplish this mix.  A “membership” is certainly more customer intimate and allows customized communication, a key component of customer intimate execution.  Again, this flows into the advertising content.  Sam’s gets to leverage the Walmart infra, so they can at the same time maintain a decent level of operational excellence.  Remains to be seen if they could do so without Walmart.

Nieman Marcus on the other hand uses a customer intimate value proposition, and their execution reflects that.  Value pricing is traded off for a high level of customization and personal service, where repeat business is very important since the number of customers this proposition attracts is smaller than the “mass” approach;  you have fewer, but each more valuable, customers.  In this model, mass media is not very effective because the audience is not mass; instead, you rely on the intimacy to Pull customers in, and much more of the Marketing budget is invested not in Advertising, but on in-store (employees, fixtures, locations) and individual communication. 

This relational or customer intimate model is the root of  ”social marketing” and why any attempt to turn online social activity into some kind of mass media advertising opportunity is a complete Paradox.  A step by step example of optimizing the relationship marketing / social model is here: Marketing Bands Series.  To optimize the social model, you divert Marketing budgets away from Mass Advertising and Push into Pull areas like Usability / Store / Interfaces / Packaging, Customer Service, and Customer Retention.

Given the above, would Nieman Marcus ever consider using Walmart’s customer retention rate as a benchmark?  I think not; this approach would make no sense at all.  The mass model can’t leverage customer retention because it’s not intimate; if you can’t act on the metric, why measure it?  This is not to say Walmart “doesn’t care” about repeat business, of course they do.  But they can’t really lever it because it’s more operationally efficient for them to use the mass approach.

That’s a very long explanation for why I dislike using industry benchmarks but many, many people don’t realize how important this idea is; it’s why on a core business model basis some companies will not be able to realize significant benefits from “going “social”.  So on the whole, I would much rather use internal benchmarks that I can improve on that are aligned with the business drivers and are controllable through my own execution.

From looking at your web site, I’d judge you a Nieman as opposed to a Walmart, so customer retention can be a powerful tool for you.  So let’s talk about measuring retention.

“Retention” is a very time-specific concept – over the course of 3 months?  A year?  Five years?  A 20% retention rate over a 5 year period and a 60% retention rate over a 3 month period might both be stunning achievements, if you know what I mean.

So, if you are able to do the analysis, I would pick some marks – 3 month, 6 month, 1 year, etc. – and see what you get for repeat buyer or retention rates.  The slope of that curve will determine where any danger points are that you might take action on.  

For example, if retention falls dramatically from 3 to 6 months, then you know that you should be watching for people who have not transacted in over 3 months, and for  those people you should craft mail / e-mail promotions designed to bring them back.

As often happens with restaurants, there’s probably a good chance that if the person is still living in the area (more on this below), the reason they are not coming back is probably  controllable – they had a bad experience.  A promotion like “We’ve missed you” or “Give us another chance” that is tightly targeted to known defectors will usually pay back quite handsomely in both the short and long term. Defected customers not only visit once on the promo but also (hopefully) have a better experience and re-engage as a repeat visitor.  If your value prop is customer intimate / social, you absolutely must invest in superior customer experience so repeat experiences are rewarding.

If you see some success with this approach, you could then fine tune the analysis to find out if the dropout has a peak in month 3, 4, or 5.  This fine tunes timing of your drop; the closer you can get to the behavior with the message the more effective the campaign will  be.  There is a “peak profitability” timing in one of these months.  

Then the program can be automated, for example: if we don’t see a transaction from this person for 120 days, drop the message.  This way, you end up mailing every month but the audience is completely different and very highly targeted each and every time.  You will find this “right message, to the right person, at the right time” approach is much more profitable than mailing all customers because it directly leverages the customer intimate value prop.

Speaking of mailing all customers, the people who are still active within this 4 month time frame are probably still loyal and you can improve overall margin by not sending these special promotions to those people until they “slip” out of the 4 month window.  There’s no reason to discount to people who are highly likely to purchase anyway.  This is the Pull part of a relationship or social  execution.  What you should be really concerned about are the people who are dis-engaging, where there has been product or service failure.

In fact, in a relational marketing scenario, there is no real need to market to these people at all, you’re basically “preaching to the choir” (example) and doing so is a waste of resources (and often margin).  You will be far better off taking the money you used to spend marketing to the choir and allocating it to in-store, core value proposition ideas.

Many marketing people (especially of the Push variety) find this difficult to understand, but there no more powerful Marketing tool than your value proposition when communicating to the active customer base.  It’s why they are coming back, your Pull is already strong with them.  Why beat them over the head with messages when they are telling you by continued transacting that they like what you are doing?  Wasteful.  (more detailed example)

Finally, in a location-based scenario such as restaurants (and since you are the CEO and not running a single store), you might consider factoring in local uncontrollable churn into any metrics you create as internal benchmarks.  

Households in different areas have different natural churn (move) rates.  Since you have stores in different states, for example, one would expect a lower retention rate from stores that have a higher natural household churn rate.  These stores might be doing very well with controllable churn (product, service) but without the household churn adjustment, they could be unfairly benchmarked “bad”.  HH churn numbers are generally available free from city / state government or the Census.

Hope that helps!

Jim

Note to blog readers: Do you see the parallels above to a lot of what is going on in online publishing / advertising / marketing?  If not, see Jonathan Mendez’s Reaping the Ads You Sow for a more direct analysis of the same concept online.  The strength of the web is in Pull, in converting demand, not Push or creating it.  Use offline for Push; that’s what it’s good at, and synch the two to optimize the entire Marketing ecosystem.

Have a question on Customer Valuation, Retention, Loyalty, or Defection? Go ahead and send it to me here. If on the topic above, you can leave a comment on the post:

Relational vs. Transactional

Do you believe human beings share certain fundamental traits that define “being human”?

If so, do you believe that human beings tend to behave in certain ways under certain circumstances?

If so, do you then believe since human behavior has these tendencies, it can often be predicted?

If so, then do you think perhaps the study of Psychology and Sociology might provide you some clues to creating successful businesses, campaigns, products, and services?  While your friends and competitors are all iterating their way into oblivion?

On the web, time and time again, we see the same themes repeating.  Yet with each introduction of a new technology, these themes tend to be treated like a new discovery, even though the theme has been well established in the past.

Norms of Reciprocity is a constant human theme.  You may know the expression of these norms as ”Sharing”.  Web old timers will probably recognize this idea as “Give, then Take” from the I-Sales discussion list as early as 1995.  In various forms, this theme goes back to the beginning of human history, all the way back to the handshake and other greeting gestures.  This same theme is embedded in countless Religions all over the world: “Do onto others as you would wish them do onto you”.  At least a couple centuries old, this idea.

Norms of Reciprocity simply means this: When you do something nice for a human being, help them in some way, this human tends to feel Gratitude towards ”the doer” and tends to do something nice back.  Gratitude drives the desire to Reciprocate, because it’s just what humans do, it’s normal, a “norm”.

Norms of Reciprocity.

The Gratitude cycle doesn’t depend on what the technology is, or if there is any at all.  If anything, technology simply extends the number of humans you can engage in reciprocal behavior with.

I first heard of this theme back in the 1970’s related to the CB radio communities, and it existed before that in ham radio.  Since then, we have been through Compuserve Forums in the 80’s, message boards as early as 1985 with The Well, then e-mail discussion groups, to hybrids like Yahoo Groups, and on into Social Media. 

And in every case, the same rules of successful interaction within these communities always applied, even though the technology was different.  No matter what communications technology the “community” uses, humans find a way to organize it with certain rules. 

And the primary driver of these rules is always Norms of Reciprocity.  Give, then Take.  The rules of successfully participating in any of these communities have not changed at all.

In fact, these reciprocity norms define the meaning of ”community”.  If a “Give, then Take” attitude is not present in a message to the community, then what you have is a message called Advertising.

Advertising has no “Give”, only “Take”.

What does all this have to do with Marketing?

In mass Advertising, it’s extremely difficult to measure the effects of a campaign at the level of Individuals.  You can measure the effects on an Audience as a whole, but not on Individuals.

But when you can measure the impact on Individuals, as you can in many forms of Direct Marketing and on much of online Advertising, now you have the ability to step through a doorway and take advantage of human behavior, including Norms of Reciprocity.

And I think this is where people are getting stuck, including the proponents of everything Social. 

These folks are trying to use Audience measurement models to define the success of (Social) Campaigns targeted to Individuals.   “Social Media” is an oxymoron; it can’t be Social and Media at the same time.

The bottom line is, if you are going to embrace a two-way Social model in Marketing, you must measure the success of this effort differently.  Impressions, reach, size of audience, none of that matters in a model where Relationships - driven by Reciprocity – are the goal.

The above metrics are one-way, broadcast advertising measures.  If “Social” or “Relationships” are to be Marketing models, what’s needed is a way to measure a 2-way exchange, a Relationship.  If it’s the Relationship that’s important, why would you use a “media metric” to measure success?  What you need is a social metric.  A  measure rooted in Psychology, one that addresses Norms of Reciprocity directly.

The question you are trying to answer in Relationship Marketing is not “how many people did I Reach”?  “Influence” or any version of Reach is a crap metric in a Social model; it’s measurement for the sake of measurement.  If it’s Reach you are pegging to, then you’re not Social, you are Media, you are All Take.  There is no Exchange in Reach; Influence is a Social metric Paradox. 

There’s nothing wrong with being Reach-based entity, but just stop calling it Social.  You’re a broadcast tower, a magazine, a newspaper.  Un-Social; Media.  Personally, I don’t think it’s a very good business model, as I said several years ago, unless it goes hyper-vertical to provide context.  That means admitting the “Social as Media” business is much, much smaller than everyone thinks it is.

But let’s say you truly want to be a Social entity or use Social techniques to faciliate Marketing.  Then the real question you need to answer in this Relationship Marketing scenario is: What is the state of my Relationships – Growing, Strong, Weakening, or Failed? 

Why?

Because unless to can define “state”, your Social Marketing efforts are not actionable and you are simply Media.  What you need to know to make Social Marketing work is this: How likely are people to interact with me in the Future?  Because if you know the answer to that question, then you can take the appropriate action against a Growing, Strong, Weakening, or Failed prospect or customer state.

That’s a Relationship.  It’s about the future, not the past.  It’s about Norms of Reciprocity; what I do for or with you today defines what you are likely to do for or with me in the future.  The past is over with; the most important issue is this: where’s the Relationship going?

The question you need to answer in a Social Marketing scenario is not “did they interact with me”, because that’s in the past and there is no Social Power in the past.  The power of Social, the value of ”Give, Then Take”, is in Tomorrow.  Right?  Potential Value.  How much Reciprocity have I earned, what is the Value of this Gratitude in the Future?

The power of Social is not in how many connections you have.

It’s understanding how to make important connections more valuable.

Fortunately, if you use metrics from Psychology rather than Media, the Value and State of your Relationships – Growing, Strong, Weakening, or Failed - are metrics that are not very difficult to measure (example).

Using these Values and understanding Reciprocity, you can then leverage Gratitude and create campaigns that not only Surprise and Delight customers but make a ton of money at the same time.

When you see how well that works, you will want to start segmenting by Relationship State instead of by demographics or other non-Social “Media Metrics” to increase profits by reducing Relationship Friction.

Once you start seeing the cause and effect of true Social or Relationship Marketing, you might even get good enough to see the value of correcting Relationship mistakes before they happen.

Social = Relationship, Relationship = Psychology, not Media.

If you want to do or be Social, then by all means, get on with it already.  There’s already a Model for all this as it applies to Marketing and this model drives profits.  The measurement of success in Social is not unknown and does not require continued mystical thought grazing.  It simply requires you to decide if you are in fact a Social entity and not in reality a Media outlet with fancy new clothes.

If you are starting up a Social entity, the phrase “Norms of Reciprocity” is your gateway to decades of research and testing on humans as Social animals.  This knowledge could save you years of iteration.

If you are already a functioning Social entity, stop gazing into that navel of yours and start publishing quantifiable Metrics from Psychology and Sociology, not Media.  You’ll soon find out whether that Social thing you are doing has Marketing value to anybody or not.

If you are a Marketer trying to leverage the Social in all of us to create and strengthen Relationships, stop looking at Social like Media and demand your vendors do the same.  

Then everybody can skip the million monkey iteration thing. 

Your thoughts on the above?

Have a question on Customer Valuation, Retention, Loyalty, or Defection? Go ahead and send it to me here. If on the topic above, you can leave a comment on the post:

Norms of Reciprocity

A conference call with a Potential Client last week jogged my memory on a couple of events that happened during the flurry of Web Analytics conferences this Spring.  Here’s a portion of the call…

PC: “We’ve tried proving the profitability of our Marketing efforts and can’t seem to get the numbers working correctly.  So Jim, what we’d like you to do is take all this data we have, and justify the Marketing decisions we’ve made by proving out the ROI.”

Jim: “I’m sorry, did you say justify?  To me, justify means “find a way to prove it works”.  Is that what you are asking me to do?  Wouldn’t it be more beneficial to analyze the results, and then optimize your Marketing based on these results?”

PC: “Jim, around here we’re pretty clear our Marketing works, and Management knows this.  But Finance is asking for some backup, some numbers to justify the spend, not to analyze it.  We don’t need analysis, we need your ‘expert credibility’ to help us out with this.”

Jim: “I see,” thinking this is not a job I’m going to enjoy.  It’s the old ‘buy an outside expert’ routine, which I detest.

PC: “Jim, the team is united behind this mission, are you on board?”

Jim: “Well, perhaps I could be on board, as long as what you want is an analysis, which may also justify the decisions you have made.  But it might not, so I just want to be clear on what…”

PC: “You  know what Jim?  I don’t feel we’re going to have a fit here, I’m getting you’re not a team player.  Thanks for your time”.  CLICK

Sigh.  I’m actually grateful they hung up, I really dislike explaining to people why I won’t work with them.

And that’s when I remember one of the most interesting moments for me during the WA conference season happened at webtrends Engage.

I was on the “Socialization of Data” panel with a great crew composed of  @jacqueswarren, @anilbatra, @johnlovett, and @bgassman.  We were talking about what web analytics might look like organizationally in the future.  Specifically, we were discussing the “Center of Excellence” concept, where all the senior analysts report to the same person, and this person typically reports directly to the C-Level.

There are many reasons this idea is a good one, but the one I often stress is relieving the pressure, explicit or implicit, on an analyst to produce a certain result from their work.  In other words, to “justify” a program rather than analyze it and get at the truth.

I said something like, ”You really don’t want analysts reporting organizationally to the group they are responsible for analyzing.  This set-up tends to create a lot of pressure on the analyst to prove a program is working by torturing the data to get a desired result.”

And about half the heads in this good-sized auditorium bobbed “Yes”.

These are the people who have been asked to change a date range, to modify a filter, to exclude a segment.  To justify a program by torturing the data into saying what someone wants it to say.

That’s sad.  Really sad.  In fact, it’s downright poisonous to the long-term health of Web Analytics (or any other analytical discipline) as a profession.  It’s a rot from within, difficult to cure.

The existence of ”justification” means the business is really not being run by the numbers.  What it means is the business continues to be run by “gut feel”, and the numbers are used to justify on the backend.

That’s not analytics, that’s a lie.

Depending on your experience, if you work in an environment like this, you might want to look elsewhere for a job, because eventually this game collapses.  It has to, you see; other people can and will get the correct numbers.

Especially in Finance.

And speaking of Finance, here’s the second most remarkable thing that happened to me on the Spring Tour.  I was talking with this web analyst who reported into Marketing.  One day, the CFO said to him, “You know what?  I think you should work directly for me.  What do you think of that idea?”

After an initial heart attack, the analyst said OK.  And he is so much happier, giddy in fact.  He loves his job again, really is fired up to get to the desk in the morning.  Why the change?

“The people in Finance get it, they understand what I have to say.  Nobody asks me to fudge the data in Finance, ever.  They just want the truth – good, bad, or otherwise.”

Funny how that works; I know exactly what he means.  If you are a profit-driven Marketer – and you can prove it - the CFO is truly your best friend.  Because a CFO gets Profit Math.

Then Web Analytics Maturity Models started getting a lot of attention due to the release of a new one from webtrends.  So I’m looking through the model and suddenly these two experiences from the Spring tour above pop into my mind.  I think:

You can have the best processes and procedures on the planet, but if you also have this Justification thing going on, if your analytical org chart is designed to fail, these Maturity Models are all just crap.  Literally.  A gigantic waste of time for everybody.

Worse than Scrap Learning.

Now, I’m not picking on webtrends here, because I like their model.  But what I don’t see in this model or the others is anything about properly Managing Analytical Cultures, like these org chart conflicts that drive Justification.  Not sure if this issue belongs in Governance, or Domain Expertise, or some other place.  I know the IT side has established “formulas” for Maturity Models, so maybe this org chart stuff doesn’t belong in the Maturity Model itself.

But this issue of reporting structure needs to at least be addressed in Maturity Model supporting documentation.  What good is it to have all the gears turning properly if the analysis itself is faulty, and drives continued poor decision making?  What kind of Maturity is that?

Speaking of chaos, if you’re in web analytics and find out your area has been targeted for downsizing but you would like to stay with the company, here’s an idea.  Head down to Finance and ask them if they would like their own web analytics person.  You might be surprised at the response.  After all, what is it most of the people in Finance do?

That’s right, Analysis.

What do you think about this issue?  Have you ever been forced to Justify?  Are you asked to run reports with “special parameters” for some programs?  To bury or exclude certain reports?

Got any good data torture stories?

Does this organizational topic belong in the WA Maturity Model?  If not, how would you handle it, where does it belong?

Have a question on Customer Valuation, Retention, Loyalty, or Defection? Go ahead and send it to me here. If on the topic above, you can leave a comment on the post:

Analyze, Not Justify

We started talking audience segmentation and content at the eMetrics Summit, and specifically the “shootout” format from the old days.  You know, 10 vendors on the stage at the same time taking questions from the audience.  Those sessions were both educational and hilarious at the same time, as the vendors side-swiped each other on topics like accuracy, how visitors are counted, cookie structures, and so forth.

But that was back when the technology was in flux, and now that issue has settled down a lot.  Braden brought up the concept of returning the “shootout format”, but more on the business side.  You know, get some practitioners, vendors, and consultants up on stage and have them thrash out stuff like:

1.  Attribution – does it really make sense to even bother with attribution at the impression / click level when there is often not a strong correlation to profit?  I mean, just because someone sees or clicks on an ad does not mean the ad had a positive effect; in fact, it may have had a negative effect.  Why not go straight to action or profit attribution, instead of using creative accounting?

2.  Why is there so much focus on real time measurement and so little “taking action” in real time?  What’s the point of reporting in real time and then having monthly meetings to take action on performance?  Why not take a gut check on the numbers at noon each day and if you are behind target, do something now?  If your org will not “act now”, then why measure what happens “now”?

3.  Does tracking page views and visits really matter in all but a few models?  If your KPI’s are worth anything, why would you look at visits or page views except “forensically”, to troubleshoot a problem revealed by the KPI’s?

4.  Why do people spend so much time measuring really small, insignificant ideas while ignoring the “big levers” they could pull that would really make a difference?  Is it possible people don’t really understand the business model they are operating in and just measure a lot of meaningless stuff ”because they can”?

The panelists could define the problem (if any) and provide potential solutions and work-arounds for these challenges.  What do you think? 

Would you like to see “shootouts” between experts on these topics?  What other business / marketing topics would you like to see shootouts or group discussions on?

Have a question on Customer Valuation, Retention, Loyalty, or Defection? Go ahead and send it to me here. If on the topic above, you can leave a comment on the post:

eMetrics “ShootOuts” We’d Like to See

The latest shiny object where Coupon Proneness comes into play is the “shopping cart recapture” program.  Mark my words, if it is not happening already, these programs are teaching customers to “Add to Cart” and then abandon it, waiting for an e-mail with a discount to “recapture” this sale – a sale that for many receiving the e-mail, would have taken place anyway. 

The best way to measure this effect is to use a Control Group.

When I hear people talking about programs like this (for example, in the Yahoo analytics group) what I hear is “the faster you send the e-mail, the higher the response rate you get”.

That, my friends, is pretty much a guarantee that a majority of the people receiving that e-mail would have bought anyway.  Hold out a random sample of the population and prove it to yourself.  There is a best, most profitable time to send such an e-mail, and that time will be revealed to you using a controlled test.  The correct timing is almost certainly not within 24 or even 48 hours.

That is, if you care about Profits over Sales, and trust me, somebody at your company does.  They just have not told you yet!

When you give away margin you do not have to give away on a sale, that is a cost.  Unless you are including that cost in your campaign analysis, you are not reflecting the true financial nature of the campaigns you are doing.  If you are an analyst, that’s a problem.

If you are using cart recapture campaigns, please do a controlled test sooner rather than later.  Because once your customers have Discount Proneness, it will be very painful to fix.

For that matter, if you are an online Marketer in a multi-channel company, you should be regularly using controls because they are the gold standard in Marketing Measurement / Campaign Attribution.

At some point, your boss will be more concerned about attributing Profit than attributing Sales.  Would be nice if your response to this question was, “Yea, we’ve been looking into that”, wouldn’t it?

If you’d like to hear more about this topic and see some example data on what these scenarios look like, you can attend this webinar:

What Online Marketers Can Teach Offline Colleagues (and vice versa)
May 19, 2009  noon ET     Jim Novo, Kevin Hillstrom, and Akin Arikan

A WAA event, open to both members and non-members.  Web analysts are not the first to grapple with multiple channels.  Traditional marketers have always had to illuminate customer behavior across stores, call center, direct mail, etc.  So, rather than reinventing the wheel in each camp, what proven methods can you teach each other?  Three different but aligned approaches on solving the multichannel puzzle, should be something for everyone here.

Have a question on Customer Valuation, Retention, Loyalty, or Defection? Go ahead and send it to me here. If on the topic above, you can leave a comment on the post:

Got Discount Proneness?

For example, did you even know there is a peer-reviewed journal called Marketing Science, which calls itself “the premier journal focusing on empirical and theoretical quantitative research in marketing”?

Whoa, say what?

This journal is published by the Institute for Operations Research and the Management Sciences, and articles are the work of premiere researchers in visitor and customer behavior from the best known institutions around the world.  In case you didn’t know, “peer-reviewed” means a bunch of these researchers (not including the authors, of course) have to agree that what you say in your article is logical based on the data, and that any testing you carried out adhered to the most stringent protocols – sampling, stats, test construction, all of it.

And, most mind-blowing of all, they show you the actual math right in the article – the data, variables, formulas, graphs – that lead to the conclusions they formulate in the studies.  You know, like this:

So, the opinions coming from Marketing Science are probably a lot more reliable than say, the average blogger in the echo chamber.  Know what I’m saying?  And here’s a surprise, the findings in these  articles often contradict what is passed off in the blogosphere as “common knowledge” by the digerati.

In case you are thinking, “Well, these lab coats can’t possibly be exploring anything that would be interesting to me“, take a look at some article titles in the Mar - Apr 2009 edition of Marketing Science:

* Website Morphing

* Real-Time Evaluation of E-mail Campaign Performance

* Optimal Bundling Strategies in Multiobject Auctions of Complements or Substitutes

* Path Data in Marketing: An Integrative Framework and Prospectus for Model Building

* Click Fraud

 Or article titles for the Jan – Feb 2009 edition of Marketing Science:

* Quantifying the Long-Term Impact of Negative Word of Mouth on Cash Flows and Stock Price

* Going Where the Ad Leads You: On High Advertised Prices and Searching Where to Buy

* Content vs. Advertising: The Impact of Competition on Media Firm Strategy

* My Mobile Music: An Adaptive Personalization System for Digital Audio Players

Tell me you want to put more faith in your RSS feed than in what these folks have to say in Marketing Science.  What blogger shows you the math?  What Research for Press Release piece passed around the echo chamber shows you the survey questions, the sample distributions, the confidence intervals?

But wait, there’s more.

The Research Committee at the WAA has been working hard on a program to open up some of these influential resources to WAA members.  You can see the beginning of those efforts here.  Expect more in the future as the effort ramps up.  The knowledge sources currently in the hopper are these:

By the way, if you are a WAA Member and would like to participate in reviewing these articles for the WAA Membership, volunteer for the Research Committee.  Contact Christopher:

christopherb “at” criticalmass.com

Now, you don’t have to agree with what folks who publish in Journals like these have to say, but as an analyst, I sure want to know what they think, see how they arrived at those thoughts.  And gut-check my own reality against them. 

Because the analyst’s mission is truth seeking, finding better truths.

Can you afford to ignore some of the most respected Marketing researchers in the world when formulating your hypothesis?

Have a question on Customer Valuation, Retention, Loyalty, or Defection? Go ahead and send it to me here. If on the topic above, you can leave a comment on the post:

Marketing Science (Journal)