Marketing Productivity Blog

This post is part of a series on control groups.  The first post is here, a list of all posts in the series here. 

Using control groups standardizes success tracking across:

Platforms
Sources
Channels

so that you begin to really understand what types of marketing create the most value.  There’s only a couple of things left you need to know to start using this gold standard of customer campaign measurement.

I would be remiss if I didn’t at least warn you once to make sure you use a true random sample of the campaign population for the control group.  The direct marketing road is littered with the bodies of those who failed to create a truly random control group for one reason or another, usually accidently, sometimes intentionally. 

For example, they sort by customer number lowest to highest then truncate sample selection before the whole population has been sampled, not realizing the lower the customer number, the longer the person has been a customer.  This creates a bias in control towards “older” customers and screws up the result.  Another common mistake is while trying to make sure the sample is random from a demographic perspective, they end up with a behavioral bias like a higher percentage of Recent buyers in Control than in Test.  There’s nothing that will make your campaign look like it sucked more than stacking Control with customers more likely to respond than those in Test!

The final issue I’d like to bring up is the ”organizational stamina” required to execute a controlled testing program. 

In large organizations, a challenge you may encounter is having other people’s campaigns “poison” your control or test groups.  The whole idea of the control is to have this group different in only one way from the test group - they don’t receive your campaign. 

What can happen is someone working with a different segmentation scheme can end up targeting portions of your test or control group, and now you don’t have a controlled test anymore - the control or test has been “poisoned”.

Just to be clear, if the test and control groups are targeted equally, then your test should still be valid, though the overall outcome might be different.  For example, let’s say you have your test and control groups and the company decides to drop a newsletter or announcement to all customers.  Since both test and control will be exposed equally to this newsletter, the incremental effects of your campaign should be preserved. 

Likewise if a national TV campaign is launched.  Your campaign might perform better overall because of the TV, but the lift you get in test versus control should be the same because the TV should affect both test and control equally.

In large organizations where many different groups access the same customer or prospect database, you can see how this poisoning of controlled tests would get to be a mess in quick order.  Without coordination, people would be stomping all over the tests by targeting a piece of a control here and a piece of test there. 

In orgs that are serious about Marketing Productivity, you do typically see a gatekeeper of some kind at the database, making sure that new list pulls do not interfere with any controlled tests that are running.  And yes, sometimes you have to wait to execute your test because there simply are not enough names to go around for the segment you want.  But this is a small price to pay compared to the total chaos of not ever knowing which marketing really works and which does not.

Clearly, there are some Marketing folks who don’t care to know how a campaign really works; “response” is just fine.  In fact, marketing chaos in the database is good for these folks.  Chaos is a fantastic barrier to accountability and the Marketers can just claim ignorance of this control group issue.  That is, until someone with a background in Business Intelligence asks why controls are not being used - and that will not be a pretty day for the Marketer.

But for the analysts out there, I really think it is your duty to start looking at the use of control groups.  Try it a few times and see what you get.  I guarantee you’ll be surprised, and the data you see will open the door to new kinds of thinking and more effective marketing programs for your customer base.

This post is part of a series on control groups.  The first post is here, a list of all posts in the series here. 

Mike Moran recently wrote about how Search Marketing is Direct Marketing.  I myself commented ”the Web is a direct marketing machine” back in 2001 when most people hated the idea of PPC marketing and thought it would never catch on.

Most of the critical breakthroughs in optimizing online marketing have been based on direct or database marketing principles that have been around for decades.  In my last post on Control Groups, I said “the insights you will get from using controls will be mind blowing.  You will begin to really understand customer behavior, and that’s the first step to creating truly game-changing customer marketing campaigns”.

I have some examples for you.

Check out this list detailing some of those insights.  Sure, they are in the form of “mistakes” but they are insights nonetheless.  See 41 Timeless Ways to Screw Up Direct Marketing by Nicholas J. Radcliffe.

The interesting thing about this list is most of these mistakes can only be identified if you are using control groups; that’s how important the concept is to customer-centric marketing.    For some mistakes on this list, you will think to yourself, “How could they ever measure that?”

The answer is one you are familiar with: repeated testing, in this case over many different industries and using many different data sets.  But you have to add controls to the test or you won’t see the effects.

Many of these mistakes are things you hear the CRM / customer-centric / CGM pundits talk about all the time, stuff like talking down to the customer, over-communicating, or being intrusive.  But these same folks never offer any conclusive proof of the financial damage these acts can cause; it’s all “gut feel”. 

How would you like to be able to prove what the damage caused by reckless marketing is really worth?

Online marketers are currently making many of these same 41 mistakes - they just don’t know it yet.  #17 and #19 are going to be very disruptive when they become widely understood.  If you want to understand more about these mistakes, a specific example is here or for a broader framework to work from, see here.

But the real question at hand is this: Will you be a driver of the next level of achievement in online customer marketing by suggesting (and eventually requiring) the use of Control Groups?

In the final post of this series, we’ll touch on two challenges with the implementation of control groups.

This post is part of a series on control groups.  The first post is here, a list of all posts in the series here. 

There are a couple of analytical culture issues I’d like to touch on with using control groups.  Control groups are the gold standard in customer marketing campaign measurement, and at some point, you will be asked to use them.  Heck, you might even get fired for not using them (think new boss comes in), as is the case at Harrahs.

Despite all this, the most obvious stumbling block is you will take a small hit on the revenue line because you’re not dropping the campaign to the control group.  I can hear it now, “But Jim, I can’t afford to take a hit on the revenue”. 

My answer to this is always the same, “You can’t afford not to take the hit, because you absolutely do not know what your true revenue generation is.”  Imagine being in the position of dramatically understating or overstating the true incremental revenue generated by your campaigns – sometimes for years and years.  This is not a pretty picture when it has to be explained.  Personally, I like to avoid that kind of thing!

So I’m just saying, you might want to mess around with control groups a bit before using them gets forced on you.  Controls are a “best practice”, and I don’t know of anyone that can really defend not using best practices.  If your company has a BI group, it’s only a matter of time before somebody over there forces the use of controls.

So how do you deal with the revenue hit?  Like much of analytics, it’s all about explaining what you are doing and why.  Instead of “gross sales”, the campaign focus becomes “sales per customer” – customer centric, if you will.  You are moving to a more customer-focused measurement system.  The goal is lift, improvement in performance, Marketing Productivity.  The tiny loss in sales from the control group is simply a cost of measuring customer marketing properly. 

And trust me, the insights you will get from using controls will be mind blowing.  You will begin to really understand customer behavior, and that’s the first step to creating truly game-changing customer marketing campaigns.

For example, often the increase in sales attribution to your campaigns from using controls will dwarf the loss in sales by not marketing to controls by a factor of 10 or more.  So while you are worrying about dropping half a percentage in campaign revenue by not using a control, you are leaving an increase of 5% in corrected revenue attribution on the table.

How’s that math working for ya?

Yes, this change will probably will be about as painful as explaining to management why you are moving from measuring hits to measuring page views, but that’s life in analytics.  When there is a better way to measure something, you should embrace it – and teach those around you why it makes more sense to measure that way.

More on the cultural issues of using control groups in the next post.

What about you?  Have you faced this “revenue drop” issue with control groups?  How did you handle it?

Information Week gives us this article: Step Up - or Step Back.  Before the Marketers in the audience click Back, I think you should read this article. 

Lead Data: From the annual meeting last month of the Society for Information Management, the percentage of CIOs and other top IT executives reporting directly to CEOs had fallen dramatically from the year-earlier survey, SIM revealed.

The premise is basically this: the “Command and Control” CIO is on the way out; these are the folks that are dropping in rank and no longer reporting to the CEO.  At the same time, we find CIO’s that are business oriented and advocates for process improvement are moving up and more of them are reporting to the CEO.

Makes sense to me.

There seems to be a lot parallels between what is going on with CIO’s and CMO’s; both are looking for a seat at the strategic table.  And both need to become more business-oriented to do it.  I think “business oriented” here is probably just a code word for “more accountable for what you contribute”.  In the case of CMO’s, this includes reaching out into the operational side of the business and finding out how operations affects the success of Marketing.

To go a step further, wanna-be CIO’s and CMO’s not afraid of an accountable orientation would do themselves a huge favor by reaching out to each other; otherwise both or either may be “absorbed into the Network“. 

This pattern playing out over in CIO-land has some lessons for those (mostly analytical) Marketers who aspire to the CMO seat.  If you do aspire to be CMO, read about the CIO’s who do report to the CEO and the business attitude that got them there - the same attitude you need.

Here’s that article link again: Step Up - or Step Back

(This post is more or less an narrative from my presentation at the 2007 Washington D.C. eMetrics / Marketing Optimization Summit)

“You know that campaign with the best response rate ever, the one with $5 million in sales?  We lost over $1 million dollars on it, according to Finance.  Something about the difference between Measuring Campaigns and Measuring Customers.”

- Me, giving my boss at HSN a piece of good news, 1991

That, my friends, was the first time I found out just how important control groups are to measuring the success of customer campaigns in an interactive, always on environment. 

The Finance department - through the Business Intelligence unit - was measuring the net profitability of the campaign at the customer level.  We (Marketing) were measuring the net profitability at the campaign level - based on response to the campaign.  The difference was close to $3 million dollars - from a $1.9 million profit using Marketing’s campaign measurement to a nearly $1 million loss using Finance / BI’s customer measurement.

The crux of this difference is always on, self-service demand, or what Kevin calls Organic Demand.  The only way to measure these customer demand effects accurately - and so the true profitability of campaigns - is with control groups.  Online, this issue is primarily relevant to e-mail marketers (customer marketing) but comes into play in lots of different ways - especially so if you have PPC or display advertising taking credit for generating sales from existing customers.

Seems like there is a lot of confusion around what control groups are and why you should care about them, and I’m hoping this post helps to clear some of that up!  But before I lose you in the details, here is why you should care about this topic:

1.  Tactically: First and foremost, if you’re not using control groups, you are most likely chronically underestimating the sales / visits / whatever KPI you generate.  “Response” is almost always lower than actual demand, because your campaigns generate sales / vists / whatever KPI you cannot track through campaign response mechanisms.  Is full credit for what you contribute to the bottom line important to you?  If so, stick around and read the rest of the post.

2.  Strategically: In a multi-platform, multi-channel, multi-source world, control groups are the gold standard in customer campaign measurement.  You will eventually be required to have a common success measurement that can be used for any situation, as opposed to success measurements “customized” for the quirks of every marketing situation that develops.

If you are not using controls, then your campaign results are always suspect.  The fact nobody has asked you yet to prove the sales you claim to generate are actually generated by your campaigns is not an excuse; that day will come.  Will you be ready?  When “prove it” is on the table, the folks using control groups win over those who are not using them every time.

3. Culturally: The concept of “variance reporting” fundamental to the control group idea is very well understood by senior management.  In fact, despite sounding complex, the control group idea is absolutely the easiest to explain to management and generates a tremendous level of confidence in what you are doing. 

This is why confidence in controlled results is so high: there are no “caveats” and no need for specialized understanding from management of different channels or technologies.  No explanations required for technological causes of error - why does this system say sales were this and this other system say sales were that?  No doubts about the source of the ROI, no questions about external effects.  Clean and simple, elegant in execution.

Interested?  OK, here we go.  Here is the idea in a nutshell.

Let’s talk a little about the idea of “incremental”, as in incremental sales or visits.  Incremental means “extra beyond normal” or what is often called “lift” in the database marketing / BI world.  The central issue is this:  if I spend money on a campaign, I want the campaign to generate incremental sales beyond what I would get if I did not do the campaign.  That’s logical, right?  Why else spend the money, if the campaign is not going to lift my sales over and above what they would have been without the campaign?

In offline retail, Wall Street is always after one KPI - called the “comps”, short for ”same store sales comparisons”.  What they want to know is for stores open at least a year, what were the sales this quarter versus same quarter last year?  That growth, or lift, is what determines how well the company is doing.  The reason is simple: if they just look at gross demand, it can be inflated by opening new stores.  These new store openings mask the true productivity of the operation, and Wall Street knows productivity is what drives profit growth in retail.  So they want to know the incremental sales versus last year of a finite set of stores open at least a year - not the sales of all stores.  In using this approach, they are controlling for the new store openings - removing the influence of them.

And that’s exactly what control groups are for - to remove the influence of any number of factors, and arrive at the true driver of the incremental change.  

When testing the effectiveness of drugs, one of the control groups is often the placebo - the people who take a sugar pill instead of the real drug.  This is done because of the placebo effect - the tendency of a person to feel better when they are taking a drug.  Why is this done?  Because the testers want to measure the real contribution of the drug - the incremental effects over and above the placebo effect.

OK?  So here is how it works in customer marketing:

1.  Choose a population to target with a campaign

2.  Take out a random sample of that population to use as control - the “control group”.  The remaining members of the population after the sample is taken out are called the “test group”.

3.  Send the campaign to the test group, and do nothing to the control group.  Measure the performance of the test versus control over time, and calculate the incremental impact on the test group of receiving the campaign.

A typical email campaign to best customers might look something like this.  Let’s say the campaign has an end date of 1 week after the drop; the customer has to react within a week to take advantage of the offer:

Respectable results for a best customer target - you do segment best customers out for different treatment, don’t you?

Here is what the same campaign probably looks like using a control group, after one week of response:

Note that 10% of targets were taken out as control; the remaining 90,000 received the campaign.

If this campaign had dropped to the entire population of 100,000, the campaign that generated $220,000 in sales really generated only $20,000 in sales, because the incremental sales impact of the campaign was only $20,000 ($.20 per e-mail) versus the control group who received no campaign.  The other $200,000 would have been generated by this customer segment without the campaign.  Follow?

Now at this point, you’re probably saying, “Hey Jim, I get it and all that but there’s no fricking way I’m going to implement this at my current job, I mean, I can’t take a hit like that in performance!” 

To which I would say:

1.  Don’t use controls until you change jobs - you’ll look like a major scientific testing hero at your next job!

2.  You don’t have all the data to make this call yet…we need to talk about what I call “halo effects”.

Halo effects are generally the unintended actions taken by the targets of the campaign.  At a basic level, it’s sales generated because of the campaign that you can’t track back to the campaign using a “campaign response” methodology.

Here’s what this campaign looks like after 6 weeks, when probably almost all the the halo effects would be included.  The numbers for each week are cumulative, they include the sales from the prior weeks:

Now that’s more like it!  If this campaign dropped to the entire population (including the control), it would have generated $295,000.

In this case, there were $75,000 in sales over and above what a “response” measurement of $220,000 shows.  These sales are coming primarily from people who did not respond to the campaign in a way you could track, but did respond to the campaign. 

We’ll dive deeper into explaining how and why this happens, plus address some of the execution and cultural aspects of using control groups in the next post.

Until then, Questions, Comments, Clarifications?

Um, following up on the post Research for Press Release, we have this gem from eMarketer and Anderson Analytics, who apparently did not even read the results of the survey they conducted.

Some highlights from this group of “Senior Marketers”:

Most important thing they are concentrating on:

Mastering the Basics

Seems unusual for Senior Marketers, to me.  

My guess: the members of MENG are not Senior Marketers, and should not be referred to as such.  Of course, nobody would pay attention to a press release about a survey on a “bunch of pukes”; this is the Source of Sample problem.

Asking “which demographic segment is most important to target” generically without supplying the product to be marketed is a ridiculous concept.  “Senior Marketers” probably wouldn’t even answer this question.

And the biggest gut-splitter: the list of “Most Important Marketing Gurus” includes Al Gore & Warren Buffet.  Now, these are both smart gents in their own ways but I’m not aware of their status as Marketing Gurus.

Of course, an alternative reality is possible: the members of MENG are Senior Marketers.  If that’s the case, I simply don’t know what to say, other than Marketing has probably already Deconstucted.  Or Imploded.  Or something worse.

You can learn a lot more from this really useful Research for Press Release (RFPR?) piece here.

Comments?

Circling back to the idea of Faulty Segmentation Logic as the root cause of “so much data, too little insight”, here’s a recent article from MyCustomer.com outlining 7 common segmentation mistakes:

Article: The seven perils of segmentation

In my experience, #1, 5, 6, and 7 are the ones that really create a lack of faith and lead to that “drowning in data” feeling.  #5 - 7 share an important commonality: the segmentation was done outside the database to be acted on, resulting in no way to tie the segmentation to the database.  This is more common than you think, and often happens when people get “survey happy”.

There’s nothing wrong with gathering survey data, but I strongly urge people to know specifically who they are surveying from a behavioral perspective (new customer, best customer, recent visitor / buyer, lapsed visitor / buyer, etc.) so you can go back and apply your new survey knowledge against specific segments.  Plus, this approach allows you to determine if people act the way they say they will act in the survey - a critical piece of insight.

Quote from peril #7: “I’ve often seen situations where firms end up with customers neatly grouped into segments… and then the marketers ask ‘now what???’”  This is the what to do with “People” as opposed to “Reach and Frequency” Marketing challenge mentioned in the previous post.

Most Marketers got their start looking at their tasks through a demographic lens via buying media, and to be fair, changing that mindset to a more behavioral or people-based view is difficult.  But the outcome is very much in synch with the current trend towards Relevance, Respect, and Relationships that so many folks are clamoring about in all the subcultures of Customer Marketing and Social Media. 

A behavior-based messaging approach can be the glue that binds all of these ideas together if you match the marketing approach with your segmentation. 

For a web analytics oriented / process view of segmentation, see Judah’s post.  The unique thing about segmentation on the web is you are often analyzing the behavior of non-customers, something many off-liners are not familiar with and presents some unique challenges.  But the same behavioral marketing concepts of Relevance, Respect, and Relationships apply. 

More on this concept and a simple model you can use to help with Marketing execution against behavior-based segmentation to come.

Poor BI; still struggling with broader adoption - as outlined by Ron in the post Four BS BI Trends (And One Good One).  So Gartner identifies BI as the “number one technology issue for 2007″ then immediately pulls out this old chestnut as BI Trend #1: There’s so much data, but too little insight.

Sigh.  

Then I get this comment by Ron Patiro asking: Besides simply not being actionable, what are some of the common pitfalls and tangles of metrics that analysts get themselves into in the pursuit of engagement?

These two ideas are closely related.  The “common pitfalls and tangles of metrics” are often the reason people get a ”so much data, but too little insight” experience.  Let’s explore these issues a bit.

The primary reason you get a “so much data, but too little insight” situation - if you have an analyst to work with the data - is indeed the actionable analysis problem, as Ron P.  points out.  But, there are at least 3 versions of the actionable analysis problem, one obvious and two not so obvious:

• Producing analysis that isn’t actionable at all
• Producing analysis that is valid but too complex to be actionable, and
• Failing to act correctly on a valid and easy to understand analysis 

And often, I find the Root Cause of these three problems (to answer Ron P’s question) to be faulty segmentation logic.  This condition in turn often is born of a situation many web analysts are familiar with by now: No Clear Objective.  But let’s leave the segmentation discussion for later and examine each of three cases above.

One cause of the “too much data, no insight” experience is producing analysis that isn’t actionable at all; it’s literally worthless and cannot be acted upon.  This is the most common vision of the actionable analysis problem – but probably not the one causing the majority of the negative outcomes.  Analysis can be “actionable” from the analysts’ perspective, but not the business perspective.  And if no actual business action takes place, no real insight is gained.

In my experience, people spend an incredible amount of time analyzing things that will never create impact.  Even if the analysis produces something that looks actionable, often the execution is impractical or financially irrelevant and so is not acted upon.  Just because you can “find a pattern” does not mean the business can do anything productive with that pattern.  Randomly “mining for gold” is one of the biggest time wasters around, and why people are often dissatisfied with the result they get from black box data mining projects.  You have to start with an actual business problem of some kind, preferably one that if solved, will increase sales or reduce costs, or no action will be taken.  Otherwise, you have simply created more data to add to the “too much data” side of the problem.

The bottom line for this slice of the problem: The intent and result of the analysis might be actionable, but unless there is a clear business case for acting, you have just contributed to the actionable analysis problem.  In other words, there is a difference between an analysis being “actionable” and having people actually act on it.

The 2nd slice of the “too much data, no insight” problem occurs when the analysis is too complex.   In Marketing at least, complexity introduces error, and probably more importantly, hinders the explanation of the analysis to people who might take action and gain insight.  If a Marketing person can’t understand the analysis, how are they going to formulate a campaign or program to address the problem, never mind get budget to act on the analysis?  Please note I’m talking about the analysis, not solving the problem itself.  Often, an elegantly simple analysis uncovers a problem that will be quite complex to solve.  These are two different issues. 

In fact, I would go as far as to say the more complex the problem is to be solved, the more elegantly simple the analysis needs to be.  The reason is this: the most complex Marketing / Customer problems are usually cross-functional in nature, and to drive success in a cross-functional project, you need rock-simple analysis that galvanizes the team without a lot of second-guessing on the value of a successful outcome.

The bottom line for this slice of the problem: An analysis might be correct and even actionable, but too complex to be acted on.  Complexity opens the analysis up to (often accurate) disbelief in the conclusion, action never takes place, so insight is lost.

The 3rd “too much data, no insight” problem is failure to translate a valid and easy to understand analysis into the correct action.  Here, we are finally moving out of the analytics side of the problem (delivering actionable analysis) and into the Business side.

Why is there failure to act correctly?  I’d submit to you it goes back to the Deconstruction of Marketing - most marketing folks simply don’t understand what to do with “people” as opposed to “Reach and Frequency”.  In other words, they can’t conceptualize how to act successfully against the individual or behavioral segment level as opposed to the nameless, faceless demographic level. 

In my opinion, this is the primary reason why demographics are so overused in customer analysis, especially online – the marketing folks simply can’t get out of that box, it’s where the “actionability” starts for them.  The problem with this thought process, as has been pointed out, is that demographics often have little to do with behavior.  Behavior predicts behavior; demographics are mostly coincidental.  Yet the analyst, looking to produce a successful project, often will allow themselves to be dragged into endless demographic segmentation that is primarily a waste of time (unless you are a media site and sell demos) and leads to false conclusions, which lead to failed or inconsistent implementation.

The bottom line for this slice of the problem: the analysis identified a problem or opportunity, but in the end, the execution against the analysis was flawed and ultimately delivered poor or no real insight.  By the way, I think this third form of failure to deliver insight is the most common – much more common than most people think.  Why?  It’s the hidden one, the one that’s not so obvious and much easier to push under the table.

So there you have it.  Three versions of the “actionable analysis” problem that lead directly to the “so much data, but too little insight” issue.  I think #3 is probably the most prevalent; a lot of analysis “fails” not because of poor analysis, but poor execution against the analysis.

What do you think?  Have you delivered a clearly actionable analysis, one that is capable of real business impact, only to have the execution against the analysis botched?

Perhaps more importantly, were you able to do anything about the botched execution?  Were you able to turn it around?  How did you make that happen?

Or, is execution not really your problem - if Marketing (or whoever) screws it up, then they screw it up?

As I said in my presentation at the eMetrics / Marketing Optimization Summit, if you want to get C-Level people to start paying attention to web analytics, you have to get into the business of predicting / forecasting.  Let’s face it, KPI’s are about the past, right?  You don’t know “Performance” until it has already happened.

But C-folks don’t really care much about what has already happened, because they can’t do anything about it.  What they really want to know is what you think will happen.  For example, ideas like “sales pipeline” - a forecast.  If you start forecasting - and you are right - you will get attention from the C-folks pronto.  The web is a great forecasting tool because it’s so frictionless; it tends to provide tangible signals before many other parts of the business.

So: Do you have any KFI’s - Key Forecast Indicators?

I have one for the Lab Store, and it tripped about 2 months ago.  It’s the Unwanted Exotic Index (UEI).

As part of the Lab Store, we run a moderated board where people who want to give up exotic pets can post the availability, and people looking for exotic pets can post requests.  Typically, the ratio of people giving them up to wanting them is about .25 - for every post looking to give an exotic up, there are 4 posts looking to adopt.

A couple of months ago, this ratio starts popping higher.  A couple of weeks ago it hit 1.25 - for every 5 posts looking to give up an exotic there were 4 posts looking to adopt.  The last time something like this happened was prior to the mini-recession of 2004, when the Unwanted Exotic Index tagged 1.0 for a short time.  After this happened, our sales got soft about 2 - 3 months later.

Why is the UEI predictive?  Let’s go through the logic - my logic, anyway!

Keeping certain types of exotic animals can be a strain on a family, both from a time and money perspective.  They can be high maintenance.  On the margin, as the economy gets tougher and people look to manage household budgets, these pets can get some scrutiny - particularly if kids have lost interest or gone off to college.  So more go up for adoption.  At the same time, requests to adopt fall, as families who might have considered an exotic pet put the “owning decision” on hold.  Taken together, these decisions cause the UEI to spike higher.  Both giving up and deciding not to own exotic pets affects Lab Store revenues “expected” in the future.  So the UEI ends up being predictive of future demand.

Makes sense to me.

Now, I’m a pretty good student of macroeconomics and pay attention to many economic indicators, especially predictive ones like the ECRI’s US Weekly Leading Index.  If you’re an analyst, you should too; economic indicators provide context for any analysis you might have to do, and clients often want to understand the impact of these external issues on their business.

As far as the Lab Store specifically, I don’t usually pay much attention to the macroeconomic cycles.  The pet business tends to be insensitive to the economic cycle; people don’t stop caring for pets as the economy wobbles up and down.  That’s why it’s such a good business - if you can find a niche.  So I don’t get too concerned when I see these predictive macroeconomic indexes forecasting a slowing economy.

However, what we have here with our Unwanted Exotic Index is a confirmation of the broader economic forecasting tools that is specific to our exotic pet business.  That makes me sit up and take notice!  Looks like our business is setting up for a repeat of the 2004 slowdown - the last time the UEI spiked like this.  Why is this important?  Because I can do something with this knowledge.  I can re-allocate and re-prioritize based on this knowledge.  For example, I can move from a “grow bigger” to a “grow smarter” mode.

And please note: this KFI has nothing to do with traffic or sales on the web site; traffic and sales are “rear view”.  By the time you see the sales slow down it will be too late to do anything about it.  And that’s why the C-folks don’t care much about web analytics reports.  

You could track an index like the UEI with a web analytics tool, but you’d have to come up with the idea first.  My point is you will probably have to look outside the usual “rear view” metrics to find one with forecasting ability.  I caution you not to substitute a “survey” for a predictive model; people’s opinions are a notoriously lagging indicator.  You’ll be up to your ears in the slowdown before people start turning bearish.

So: Do you have any KFI’s - Key Forecast Indicators?  Tell us about them. 

If you don’t have any KFI’s, now is the time to start looking for them.  What can you see now that predicts what will happen in the future?  Think about the business, think about the data sources, and put together a bunch of different ideas.  Track them back a couple of years and post them monthly going forward.  You’re bound to find something predictive.  Perhaps something about posting, like the UEI.  Recommendations / comments as a percent of visitors or something like that.

If you’re stuck, start with a simple “engagement” idea - percent visitors / members / customers who visited / logged in / bought in the past 90 days.  If this percentage is falling, so will your business in the next 3 - 6 months.  If your business has a lot of seasonality in it, look to year-over-year comps of the same metric.

If you’ve never played this game before, you won’t have proof your KFI’s work until after the business is in the soup, but you’ll be ready with accurate and actionable KFI’s the next time around!

Looks like 1-to-1 Magazine has decided to unlock some of their archives, maybe releasing them to search after the next issue is published?  Who knows, but there was an important article on Failure published last month that is worth a read.

One of the major challenges the Analytical Culture faces is Fear of Failure; it’s just so uncool to fail in many companies today.  Yet some of the most spectacular wins often come after spectacular failures, and we have to teach managers that without Failure, there is no Learning Process.  Do it like they do at 3M and IBM, using the real stories of how failure went unpunished and was ultimately rewarded. 

You want the Analytics to free people, not have them seek out least common denominator “safe harbors” that have (perceived) immunity from failure.  I’m not sure many folks get how important this cultural issue is; if you don’t address it, the Analytics can actually make you worse off as people avoid risk by satisficing.

Check out the article here, more from me on this topic here.