Category Archives: Marketing / Tech Interface

Reporting versus Analysis: The “Actionable” Debate

Analytical Culture, Marketing / Tech Interface, Web Analytics

Gary Angel and Eric Peterson have been having a great exchange surrounding the definition of KPI’s, and more specifically, the requirement that they be actionable.   Gary started out with the position the “criteria of actionability is unsound in almost every way” but I think both he and Eric have resolved in the middle somewhere – it’s really about context.  Gary is right, to take any metric “naked” at face value without surrounding context is simply not good analytical practice.  But I would argue (and I think Eric agrees) that to build a KPI in the first place, you must already have the required context, or you don’t have a KPI.  So that leaves us with “how you define a KPI” as (I think) the final resting point, and there really isn’t anywhere to go after that.  Your comments on my analysis welcome.

However, I think the ideas Gary has exposed run deeper than just the KPI discussion.  The situation Gary is addressing – making sure people really understand that every metric requires business context to be functional – requires attention because web analytics is a very fast growing field with a lot of brand new people in it who may have not been exposed to proper analytical training. Or, not challenged to do any real analysis by weak managers.

These new people frequently don’t understand the difference between Reporting and Analysis.  A “Reporting” mentality (provide data) leads to the improper use of analytical ideas like KPI.  Analysis (provide insight) would automatically take into account a lot of other factors, as Gary has suggested.  Knowing all those factors (because you are doing real analysis), you can certainly take movements in a KPI as actionable.  As Eric says, that “action” is often a more focused analysis of some kind.  KPI’s are really just “tripwires” that alert you to a problem or opportunity that requires further analysis.

My concern (and in the end, I think Gary’s) is that often the Reporting mentality is Robotic and that the reaction taken to change in a KPI might be equally Robotic if you don’t have the proper context.  What often happens in Pay-per-Click testing is a great example of this, and a lot of the multivariate stuff people are now addicted to is an extreme example. 

You can look at conversion rates, make changes to landing pages, and try to optimize the “Scenario”.  This is Reporting, not Analysis.  Can you provide insight into why the changes you made worked?  For example, can you explain the improvement in terms of Psychology or Consumer Behavior?  Usability?  If so, that would be Analysis, and the answers would be applicable to a wide range of other challenges on the site.  Without knowing why the changes worked, you are left with simple Reporting that applies to only a single specific Scenario.  Nothing was really learned here.

Take this same idea to the extreme, and you get what often happens in multivariate testing.  You can certainly run a multivariate test on 5 variables at the same time, and find a “winning combination”, but this is Reporting, not Analysis – in fact, it’s black-box reporting in the extreme.  For example, how do you know that you chose the 5 most important variables to optimize?  How do you know the options you chose for each variable are the most powerful?  Isn’t it just as likely that the final optimization you achieved is suboptimal, a local maximum, as it is the solution is truly optimal? 

In other words, isn’t it possible that what you have created with the robot is better than you had, but is not even close to being the best it can be?

Dear Reader, you’re asking, why should I care about this Reporting versus Analysis issue?  Because here is what will happen without real Analysis: you are going to “hit the wall”.  One day, there will simply be nothing left you can do to improve on what you have done.  Reporting is only going to take you so far.  Frustrated, you will probably Analyze the situation and realize you have “optimized” yourself into a corner by taking something that was fundamentally broken in the first place and making it better than it was.  You can’t make it any better unless you wipe it out and start again.  That’s a huge waste of resources, right?

See CRM if you need an example of what can happen when you automate worst practices.  And they’re going to fix it 8 years later by bolting on Business Intelligence?  Um, shouldn’t the Analysis have come first?

The Deconstruction of Marketing

Analytical Culture, Analytics Education, Marketing / Tech Interface

Seems to me these days “Marketing” is being deconstructed into a bunch of pieces.  When I was coming up through the ranks, Marketing included Customer Service plus all the stuff now called CRM, Customer Experience, and all the related ideas.  The person in charge of Marketing was in charge of all these things.  It made so much sense to manage a business this way, because to control your fate as a Marketer, you had to control or at least influence all the customer touchpoints.  So why are these responsibilities being split off into little sub-cultures today?

The answer is they’re not really; that’s just the way it looks to me, because every industry I worked in for 25 years was rich with customer data and we used that data to prove why it made so much sense for Marketing to be in charge of all these aspects of the company interface with the customer.  We proved time and time again that by exerting cross-silo influence where the customer was involved, Marketing could generate much improved profitability.  Every Marketing program worked even harder towards generating profits when Marketing got all the silos aligned.

So I guess it just looks to me (and some other data-driven Marketing folks) like these functions are being split out of Marketing.  The reality is that many companies never had any of these data-driven functions before, and when they start getting access to customer data, they created these areas as new entities.  The question: why not create them under Marketing?  This approach sure would have saved a lot of trouble in CRM, for example. 

And I suspect the answer is the Marketing folks took one look at this new data-driven world with the associated need to have a basic understanding of technology issues, and said, “No thanks, I’ll stick to Advertising and PR”.  And as Marketers let go of / failed to capture control of these key operational touchpoints with the customer, they essentially devolved Marketing from a strategic C-Level force into “MarCom”.

And that’s a real shame.  This splintering of Marketing Management by technological issues is a waste of time at best and a long term problem at worst.  Ultimately, after we go through all this CRM and Chief Experience Officer stuff and whatever else you want to call it (seems like a new name every day), people will realize that all of this belongs in Marketing.  And then we’ll just need some brave Marketing folks who think they can handle it to step up to the plate and really make it work.  If you’re a mid-level MarCom person and want to start preparing for this transition, start making some friends in Finance, Technology, and Customer Service.  Find out what it is that keeps them awake at night, and think about how Marketing could help solve their problems.

And to jump-start your brain towards making Marketing decisions based on customer data rather than using nameless, faceless GRP’s, try taking a look at the business side of web analytics.  You’ll be amazed at how much of it transfers directly to Data-based / CRM / Customer Experience Marketing.  Why?  Because the web analytics community has decided best practices require a cross-functional team approach with a focus on Customer Experience and a requirement to examine the Financial implications of actions taken.  Web analytics teams are a functioning microcosm of what Marketing used to be, and what it should be in the future.

I’m not clear on the Chief Customer Officer concept…

DataBase Marketing, Marketing / Tech Interface

I don’t have any problem with the direction Jeanne Bliss provides regarding how to become the “customer champion” in your company, especially the idea of aligning with the CFO and CIO.

What I’m trying to figure out is why this is not Marketing’s job; seems to me the CMO should be the Chief Customer Officer, complete with the ultimate responsibility for Customer Service.  Otherwise, it seems like this CCO position is just an excuse for people keeping their heads in their own silos and letting somebody else worry about cross-functional processes, customer experience, and defects.

In other words, do we really need a unique exec to be able to create / enforce / enable cross-silo functionality on the “soft” side (marketing, service, some fulfillment) of the business?  After all, the CIO and CFO operate across all the silos, why can’t the CMO?  As Ron said in his excellent piece What Marketers Should Learn From IT, the CMO now needs to get involved in the whole business and act cross-functionally to be successful, as the folks in IT have learned.

I guess the answer is probably that “Marketing” has been redefined over the years and has somehow lost the strategic seat at the table, morphing downward into “MarCom”.  This has not happened at all companies – I can tell you at most truly data-driven companies, the CMO is the Chief Customer Officer, because these folks / the company understand how the totality of the customer experience affects Marketing Productivity.

Perhaps the answer to Kevin’s question on what happens to the Marketing folks in the catalog business as the web takes over is this: they become Chief Customer Officers or consultants to them like Jeanne Bliss, formerly of Lands’ End.  After all, they already know how to do the Chief Customer Officer job.