Archive for the ‘Analytical Culture’ Category

“Missing” Social Media Value

Wednesday, October 12th, 2011
I have no doubt there is some value in social beyond what can be measured, as this has been the case for all marketing since it began ;)  The problem is this value is often situational, not too mention not properly measured using an incremental basis (as you point out).
For example,  to small local businesses who do no other form of advertising, there is a huge amount of relative value to using social media, versus no advertising at all.  Some advertising is much better than none, and since it’s free, the incremental value created by (properly) using social is huge.
On the other hand, I wonder why social analysis seems to forget that people have to be aware of you to “Like” you in the first place.  Further, it seems unlikely a person would “Like” a brand or product if they have not already experienced it, and are already a fan.  If this is not true, if people “Like” a company even thought they do not (paid to Like?), then the problems with social go way beyond analysis…
But if true, , the number of “Likes” doesn’t have as much to do with awareness as it does with size of customer base, and is much more aligned with tracking customer issues (retention, loyalty) than anything to do with awareness / acquisition.
Add the fact many companies are running lots of advertising designed to create awareness, and the incremental value of social as a “media” may be close to zero, or at least less than the cost to analyze the true value of it.
And this last, really, is the core of the issue.  It’s simply not possible to measure “all” the value created by any kind of marketing, and there are hugely diminishing returns as you try to capture the last bits.  I think it’s quite possible the optimism for “value beyond what can be measured” is less than the cost of measuring it *if* people keep looking in the awareness / acquisition field.
Folks who want to find this “missing” social value should start doing customer analysis, and look in the “retention / loyalty” area, where the whole idea of social is a natural, rather than a forced, fit.

Has to be There

I find it really interesting that whenever there is a discussion of measuring the value of social media, there’s such a bias towards believing there is value in social beyond what can be properly measured.  See the comments following this post by Avinash for a good example.  Speculation is fine, but the confidence being expressed that a new tool or method will uncover a treasure trove of social media value seems un-scientific (as in scientific method) at best.

I don’t doubt there is some value in social media beyond what can be measured, as this has been the case for all marketing since marketing measurement began.  These measurement problems are not new to social either:  Marketing value created is often situational, it depends on the business model and environment.  What works in one situation may not work in another.

For example:

To small local businesses who do no other form of advertising, there is a huge amount of relative value to using social media, versus no advertising at all.  Social advertising is much better than none, and since it’s free, the incremental value created by (properly) using social is huge.  It’s also really easy to measure the impact and true value, since the baseline control is “no advertising”.  Lift, or actual net marketing performance, can be pretty obvious in his case.

On the other hand, many companies are running lots of advertising designed to create awareness, and the incremental value of social as a “media” may be close to zero for these companies, or at least less than the cost to analyze the true value of it.  Possible explanation:  Social events such as “Likes” or comments are simply representations or affirmations of awareness already created by other media, so by themselves, create little value.  In other words, events such as Likes might track the value of other media spending, but may not create much additional marketing value.

(more…)

All Talk, No #Measure

Friday, March 11th, 2011

Hypocrisy in Web Analytics?

Before every eMetrics (I’ll be in San Fran teaching Basecamp, at the Gala, etc.), I try to ask myself, what is the most critical issue facing the web analyst community right now?  Then, at the show, I ask everyone I run into what they think about this issue.

There’s lots of issues to choose from.  Career path I think is a big area of discussion, given the mergers in the space and trend towards outsourcing.  Then there’s the “we don’t get no respect” thing; senior management doesn’t seem to listen / understand / act on the information provided.  And one of my favorites from the past is still out there, data torture - people being pressured to manipulate data to reach a predetermined analytical outcome.

But seems to me, more important at this juncture is trying to resolve why there is so much written about the importance of “the customer” but very little measurement at the customer level.  Think about it.  Customer experience, customer centricity, the entire social thing, it’s all about customers.

But when folks wants to trot out “proof” that this or that approach is the road to the promised land, they analyze impressions, visits, clicks, etc.  Visitor-level stuff.  Does that seem like the correct approach to you?  Seems to me, if you want to provide knowledge about customers, you should measure customers.

(more…)

But What is an Impression Worth?

Tuesday, March 8th, 2011

Seems like coming up with a value for social media has become a cottage industry, for example, $3.60 Facebook Fan Valuation Is Just the Tip of the Iceberg.  These values are often derived from what is paid for online media.  So you have to ask, if someone is basing the value of a Facebook fan on the value of impressions generated, what is the real value of those impressions?  Because unless this is known, the whole framework is faulty.

Just because you pay $5 / CPM for impressions, does not mean they are worth $5 / CPM, does it?  Do people really still have that kind of mentality?  Is the price of the media equivalent to its value?

For example, I’m sure you have heard of multi-million dollar campaigns that generate very little lift in sales.  Happens frequently in fast food, for example.  What is the value of that media?  Is it the millions paid?

What really blows my mind about this approach is it’s so offline, so old school PR. Do the folks who put forth these kinds of models believe nothing has changed in 50 years?  What happened to the whole rap of online being “different”, that you can’t measure it like offline, blah blah.

Except when it’s convenient to do so?

If you want to know the value of a Facebook fan, why not measure the value of a Facebook fan?  Because it’s hard, and would require organizational discipline?  Too bad.   Substituting the kind of models used in the example above for actually measuring the value of a Facebook fan is misleading at the very best.

(more…)

When Does a Visitor Need a Coupon?

Friday, December 17th, 2010

The following is from the November 2010 Drilling Down Newsletter.  Got a question about Customer Measurement, Management, Valuation, Retention, Loyalty, Defection?  Just ask your question.  Also, feel free to leave a comment and I’ll reply.

Want to see the answers to previous questions?  Here’s the blog archive; the pre-blog newsletter archives are here.

Q: First off, I very much appreciate you sharing all this wonderful content on your blog and conferences such as eMetrics.

A: Thanks for that!

Q: My question is a simple one, but I think the answer may be hard: When does a visitor “need” a coupon?  *Need* defined as: visitor would not have placed an order unless presented with the coupon.

A: Hmmm…methinks we’re going to have to define a few concepts and be clear on the goals to make sure we are nailing this down… visitor versus customer, sales versus profit, etc.  In other words, answer is not hard, but could be complex without defining context.

Q: It’s still a mystery to me why so many retailers seem more than willing to hand over all their margins to Groupon or give coupons to basically all visitors.  I am curious whether you would approach this question using  observational data (eg web analytics) or experiments (eg AB testing), or both.

A: Right – is a mystery to me too!

There are certain situations where this approach might be appropriate, but the problem with much web “marketing” (which often is really just advertising without much thought about marketing) is often there is success in a narrow or special situation.  Then the pundits jump on and say “if you’re not doing this you are stupid”, regardless of the business situation and / or without recognizing the special circumstances that are driving success.  This is all the real Marketing stuff people leave out; understanding why it works, under what circumstances, for which segments, involving which products.

(more…)

LTV, RFM, LifeCycles – the Framework

Friday, June 18th, 2010

The following is from the May 2010 Drilling Down Newsletter.  Got a question about Customer Measurement, Management, Valuation, Retention, Loyalty, Defection?  Just ask your question.  Also, feel free to leave a comment and I’ll reply.

Want to see the answers to previous questions?  Here’s the blog archive; the pre-blog newsletter archives are here.

Q: I visited your website because I am trying to understand how to develop a customer LifeTime Value model for the company that I work at.  The reason is we are looking at LTV as a way to standardize the ROI measurement of different customer programs.

Not all of these programs are Marketing, some are Service, and some could be considered “Operations”.  But they all touch the customer, so we were thinking changes in customer value might be a common way to measure and compare the success of these programs.

A: Absolutely!  I just answered a question very much like this the other day, it’s great that people are becoming interested in customer value as the cross-enterprise common denominator for understanding success in any customer program!

If I am the CEO, I control dollars I can invest.  How do I decide where budget is best invested if every silo uses different metrics to prove success?  And even worse, different metrics for success within the same silo?

By establishing changes in customer value as the platform for all customer-related programs to be measured against, everyone is on an equal footing and can “fight” fairly for their share of the budget (or testing?) pie.  By using controlled testing, customers can be exposed to different treatments and lift in value can be compared on an apples to apples basis – even if you are comparing the effect of a Marketing Campaign to changes in the Service Center.

(more…)

Inside WAA Certification: Any Questions?

Friday, April 16th, 2010

The WAA has published a lot of info about the new WAA Certification Exam; you might want to first read the FAQ and take a look at the application information and Exam Handbook for the organizational details, and you can see sample questions from the Test at the bottom of the page here.  But something I can just about guarantee about the Certification – no matter how much info the WAA publishes about it, many people will still have questions!

So here, I will attempt to answer other kinds of questions I think people might have based on my discussions with WAA members.

Update: The WAA has answered many Certification questions here.

However, I’m going to approach this topic a bit differently than most of the published documentation – from a Product / Marketing perspective, rather than an Educational / WAA POV.  I can do this because (if you don’t know) I have worn all the hats on this project – developer, marketer, WAA project owner – and I think it might be helpful to tell the business story of the WAA Certification, from the bottom up.

And if you have other questions, feel free to leave them in Comments and I will do my best to answer them!

(more…)

Tortured Data – and Analysts

Tuesday, February 9th, 2010

Fear and Loathing in WA

You may recall I wrote last year about the explicit or implicit pressure put on Analysts to “torture the data” into analysis with a favorable outcome.  In a piece called Analyze, Not Justify, I described how by my count, about 50% or so of the analysts in a large conference room admitted to receiving this kind of pressure at one time or another.

Since then, I have been on somewhat of a personal mission to try to unearth more about this situation.  And it seems like the problem is getting worse, not better.

I have a theory about why this situation might be worsening.

Companies that were early to adopt web analytics were likely to already have a proper analytical culture.  You can’t put pressure on an analyst to torture data  in a company with this kind of culture – the analyst simply will not sit still for it.  The incident will be reported to senior management, and the source of “pressure” fired.  That’s all there is to it.

However, what we could be seeing now is this: as #measure adoption expands, we find the tools in more companies lacking a proper analytical culture, so the incidents of pressure to torture begin to expand.  And not just pressure to torture, but pressure to conceal, as I heard from several web analysts recently.

(more…)

Control Groups in Small Populations

Friday, February 5th, 2010

The following is from the January 2010 Drilling Down Newsletter.  Got a question about Customer Measurement, Management, Valuation, Retention, Loyalty, Defection?  Just ask your question.  Also, feel free to leave a comment and I’ll reply.

Want to see the answers to previous questions?  Here’s the blog archive; the pre-blog newsletter archives are here.

Q: Thank you for your recent article about Control Groups.  Our organization launched an online distance learning program this past August, and I’ve just completed some student behavior analysis for this past semester.

Using weekly RF-Scores based on Recently and Frequently they’ve logged in to courses within the previous three weeks, I’m able to assess their “Risk Level”– how likely they are to stop using the program.  We had a percentage who discontinued the program, but in retrospect, their login behavior and changes in their login behavior gave strong indication they were having trouble before they completely stopped using it.

A: Fantastic!  I have spoken with numerous online educators about this application of Recency – Frequency modeling, as well online research subscriptions, a similar behavioral model.  All reported great results predicting student / subscriber defection rates.

Q: I’m preparing to propose a program for the upcoming semester where we contact students by email and / or phone when their login behavior gives indication that they’re having trouble.  My hope is that by proactively contacting these students, we can resolve issues or provide assistance before things escalate to the point they defect completely.

A: Absolutely, the yield (% students / revenue retained) on a project like this should be excellent.  Plus, you will end up learning a lot about “why”, which will lead to better executions of the “potential dropout” program the more you test it.

(more…)

Relational vs. Transactional

Friday, October 2nd, 2009

The following is from the September 2009 Drilling Down Newsletter (original title:  Customer Retention for Restaurants).  Got a question about Customer Measurement, Management, Valuation, Retention, Loyalty, Defection?  Just ask your question.  Also, feel free to leave a comment.

Want to see the answers to previous questions?  Here’s the blog archive; the pre-blog newsletter archives are here.

Q:  I am hoping you can help answer a question for our team.  By way of introduction, I am the CEO of XXXX.  We are a specialty retailer / restaurant of gourmet pizza, salads and sandwiches.  We would like to know  restaurant industry averages (pizza industry if possible) for customer retention – What percentage of customers that have ordered once from a particular restaurant order from them a second time?  I am hoping with your years of expertise and harnessing data you may be able to assist us with this question.  Look forward to hearing from you.

A:  Unfortunately, in those said years of experience, I have found little hard information on customer retention rates in QSR and restaurants in general (if anyone has data, please leave in Comments).  It’s just the nature of the business that little hard data, if collected, is stored in such a way that one can aggregate at the customer level.  The high percentage of cash transactions doesn’t help matters much; there’s a lot of data missing.

Over the years, sometimes you see data leak out for tests of loyalty programs, and of course clients sometimes have anecdotal or survey data, but this is not much help in getting to a “true” retention rate.  More often than not you discover serious biases in the way the data was collected so at best, you have a biased view of a narrow segment.  Often what you get is a notion of retention among best customers, or customers willing to sign up for a loyalty card, but not all customers.  And the large “middle” group of customers is where all the Marketing leverage is.

What to do about this predicament?  

There are really two issues in your question; the idea of using industry benchmarks when analyzing customer performance, and the measurement of retention in restaurants.

(more…)

Norms of Reciprocity

Friday, June 26th, 2009

Social Marketing Doesn’t Rely on Social Media

Do you believe human beings share certain fundamental traits that define “being human”?

If so, do you believe that human beings tend to behave in certain ways under certain circumstances?

If so, do you then believe since human behavior has these tendencies, it can often be predicted?

If so, then do you think perhaps the study of Psychology and Sociology might provide you some clues to creating successful businesses, campaigns, products, and services?  While your friends and competitors are all iterating their way into oblivion?

On the web, time and time again, we see the same themes repeating.  Yet with each introduction of a new technology, these themes tend to be treated like a new discovery, even though the theme has been well established in the past.

Norms of Reciprocity is a constant human theme.  You may know the expression of these norms as ”Sharing”.  Web old timers will probably recognize this idea as “Give, then Take” from the I-Sales discussion list as early as 1995.  In various forms, this theme goes back to the beginning of human history, all the way back to the handshake and other greeting gestures.  This same theme is embedded in countless Religions all over the world: “Do onto others as you would wish them do onto you”.  At least a couple centuries old, this idea.

Norms of Reciprocity simply means this: When you do something nice for a human being, help them in some way, this human tends to feel Gratitude towards ”the doer” and tends to do something nice back.  Gratitude drives the desire to Reciprocate, because it’s just what humans do, it’s normal, a “norm”.

Norms of Reciprocity.

(more…)