Archive for the ‘Analytics Education’ Category

All Talk, No #Measure

Friday, March 11th, 2011

Hypocrisy in Web Analytics?

Before every eMetrics (I’ll be in San Fran teaching Basecamp, at the Gala, etc.), I try to ask myself, what is the most critical issue facing the web analyst community right now?  Then, at the show, I ask everyone I run into what they think about this issue.

There’s lots of issues to choose from.  Career path I think is a big area of discussion, given the mergers in the space and trend towards outsourcing.  Then there’s the “we don’t get no respect” thing; senior management doesn’t seem to listen / understand / act on the information provided.  And one of my favorites from the past is still out there, data torture - people being pressured to manipulate data to reach a predetermined analytical outcome.

But seems to me, more important at this juncture is trying to resolve why there is so much written about the importance of “the customer” but very little measurement at the customer level.  Think about it.  Customer experience, customer centricity, the entire social thing, it’s all about customers.

But when folks wants to trot out “proof” that this or that approach is the road to the promised land, they analyze impressions, visits, clicks, etc.  Visitor-level stuff.  Does that seem like the correct approach to you?  Seems to me, if you want to provide knowledge about customers, you should measure customers.

(more…)

Awareness versus Persuasion

Wednesday, September 23rd, 2009

In the early days of Home Shopping Network (live TV, not online), we were doing some ethnographic research and started to find “physical clusters” of customers – neighbors or people who worked together.  For example, one of these groups was nurses at hospitals,  especially nurses  who worked the night shift.

We looked for the most active member of the cluster (our “thought leader”) and asked them if they would help us with a “member get a member” program.  Would they be willing to distribute discount coupons to their friends, especially ones who were not already customers?  Time after time, the answer was:

“Honey, all my friends are already customers of yours”.

We launched the program anyway, because it was a pet project from upstairs  – I was a junior marketer at that point so I couldn’t kill it ;)  The program never, ever worked, no matter how hard we tried.  It generated very few new customers while giving lots of discounts to people who were already active buyers.  Basically,  the cost of those discounts overwhelmed the value of the new customers generated.

Apparently a similar thing happens online with Social marketing.

As part of a WAA program that reviews academic research for WAA members, I was able to take a look at a paper titled:  Firm-Created Word-of-Mouth Communication: Evidence from a Field Test by David Godes and Dina Mayzlin.

(more…)

eMetrics “ShootOuts” We’d Like to See

Friday, May 22nd, 2009

I was in Vancouver for a presentation to CAUCE [kay-yoose, thanks Raquel] and was able to grab a quick dinner with fellow WAA BaseCamp stakeholders Andrea Hadley, Raquel Collins, and Braden HoeppnerWe’re rolling out a new 2-day format for BaseCamp and got to talking about web analytics education in general. 

We started talking audience segmentation and content at the eMetrics Summit, and specifically the “shootout” format from the old days.  You know, 10 vendors on the stage at the same time taking questions from the audience.  Those sessions were both educational and hilarious at the same time, as the vendors side-swiped each other on topics like accuracy, how visitors are counted, cookie structures, and so forth.

But that was back when the technology was in flux, and now that issue has settled down a lot.  Braden brought up the concept of returning the “shootout format”, but more on the business side.  You know, get some practitioners, vendors, and consultants up on stage and have them thrash out stuff like:

1.  Attribution – does it really make sense to even bother with attribution at the impression / click level when there is often not a strong correlation to profit?  I mean, just because someone sees or clicks on an ad does not mean the ad had a positive effect; in fact, it may have had a negative effect.  Why not go straight to action or profit attribution, instead of using creative accounting?

(more…)

Marketing Science (Journal)

Friday, April 24th, 2009

As I said in the Heavy Lifting post, I think the Web Analytics community is becoming increasingly insular and should be paying more attention to what is going on outside the echo chamber in Marketing Measurement.  I also think the next major leaps forward in #wa are likely to come from examining best practices in other areas of Marketing Measurement and figuring out how they apply to the web.

For example, did you even know there is a peer-reviewed journal called Marketing Science, which calls itself “the premier journal focusing on empirical and theoretical quantitative research in marketing”?

Whoa, say what?

This journal is published by the Institute for Operations Research and the Management Sciences, and articles are the work of premiere researchers in visitor and customer behavior from the best known institutions around the world.  In case you didn’t know, “peer-reviewed” means a bunch of these researchers (not including the authors, of course) have to agree that what you say in your article is logical based on the data, and that any testing you carried out adhered to the most stringent protocols – sampling, stats, test construction, all of it.

And, most mind-blowing of all, they show you the actual math right in the article – the data, variables, formulas, graphs – that lead to the conclusions they formulate in the studies.  You know, like this:

(more…)

Webcast on Web Intelligence 11/19

Tuesday, November 18th, 2008

Speaking of Web Intelligence, if you are interested in experiencing what the world of web analytics looks like when it meets Business Intelligence, the WAA and our Certificate partner for Web Intelligence, UC Irvine, are doing a Free webcast on this topic. 

Jim Humphrys has the research on salaries and demand in the sector, Shaina Boone of Critical Mass is the practitioner who has both taken the Certificate classes and is applying this knowledge in the real world, and Bernie Jeltema is a UCI Instructor for the Certificate classes.

Here’s the official description:

UCI Webinar: Certificate in Web Intelligence
Wednesday 19-Nov-08 2:30 PM to 3:30 PM EDT

Web Intelligence is a combination of web analytics and business intelligence. As companies expand their reach into the global marketplace, the need to analyze how customers use their web sites to learn about products and make buying decisions is becoming increasingly critical for survival and success.  Wondering how to position yourself for these career opportunities and how specific coursework can be valuable?  This planning session will provide pre-registration educational and career advancement advisement. Also learn more about the web intelligence certificate program, courses being offered in upcoming quarters, and career planning resources available through the UC Irvine Extension and the University of British Columbia, Continuing Studies

  • Jim Humphrys, WL Gore, co-chair, WAA Research Committee
  • Shaina Boone, Critical Mass
  • Bernie Jeltema, Instructor in Business Intelligence, consultant in field
  • To register visit: http://unex.uci.edu/certificates/it/web_intel/

    On this page, you can either sign up to “Stay Informed” about the program (green bar) or Register for the webcast in the box below this bar, which is called Web Intelligence Education Planning Session.

    See Ya @ eMetrics

    Thursday, October 16th, 2008

    I’ll be speaking at the eMetrics Marketing Optimization Summit on Wednesday, October 22 at 1 PM after lunch with the Summit Advisory Council.  How is it that I get scheduled in that “after Council” speaking slot every year?  Jim Sterne must not want me hanging with the Council too long…

    I’ll be speaking about LifeCycle analysis and providing “how to act on the analysis” for the Marketing side.  If you are being asked to cut back Marketing budgets, LifeCycle analysis is a great way to understand the Financial ramifications of Marketing budget cuts and start getting Predictive.

    Coming in on Tuesday so will miss the WAA event on Sunday for the first time.  On the flip side, I will be there through Friday afternoon ’cause I am presenting at the WAA Board meeting. 

    So, for the first time since probably 2004, I will actually be there when the shindig closes.

    Who’s doing what Thursday night?

    Interview-Podcast w/ Jim Novo

    Friday, February 1st, 2008

    Friend and fellow blogger Alan Rimm-Kaufman spent some of his valuable time asking my opinion on various online marketing issues in a far-ranging interview and podcast.

    We met in person for the first time doing a presentation together at the DMA show in Chicago this fall, and because he used to work at Crutchfield – a truly customer-driven remote retailer – we share some experiences and beliefs.

    For those of you who might be wondering where a lot of the Marketing Productivity ideas I post here come from, this interview-podcast is probably a pretty good backgrounder.  We talk about a lot of stuff, including:

    Monetizing customer experience

    Importance of Control Groups / Source Attribution

    Multichannel Marketing Strategy

    LifeCycle Contact Strategy versus Calendar-based

    Retail Business Models / Lab Store

    Search box or not? / Serendipity

    How to tell if online customers are really engaged – without web analytics

    Here’s another link to the Interview-Podcast.  Enjoy! 

    That was lots of fun, thanks Allen!

    KFI’s: Key Forecast Indicators

    Sunday, November 11th, 2007

    As I said in my presentation at the eMetrics / Marketing Optimization Summit, if you want to get C-Level people to start paying attention to web analytics, you have to get into the business of predicting / forecasting.  Let’s face it, KPI’s are about the past, right?  You don’t know “Performance” until it has already happened.

    But C-folks don’t really care much about what has already happened, because they can’t do anything about it.  What they really want to know is what you think will happen.  For example, ideas like “sales pipeline” – a forecast.  If you start forecasting – and you are right - you will get attention from the C-folks pronto.  The web is a great forecasting tool because it’s so frictionless; it tends to provide tangible signals before many other parts of the business.

    So: Do you have any KFI’s – Key Forecast Indicators?

    I have one for the Lab Store, and it tripped about 2 months ago.  It’s the Unwanted Exotic Index (UEI).

    As part of the Lab Store, we run a moderated board where people who want to give up exotic pets can post the availability, and people looking for exotic pets can post requests.  Typically, the ratio of people giving them up to wanting them is about .25 - for every post looking to give an exotic up, there are 4 posts looking to adopt.

    A couple of months ago, this ratio starts popping higher.  A couple of weeks ago it hit 1.25 – for every 5 posts looking to give up an exotic there were 4 posts looking to adopt.  The last time something like this happened was prior to the mini-recession of 2004, when the Unwanted Exotic Index tagged 1.0 for a short time.  After this happened, our sales got soft about 2 – 3 months later.

    Why is the UEI predictive?  Let’s go through the logic – my logic, anyway!

    Keeping certain types of exotic animals can be a strain on a family, both from a time and money perspective.  They can be high maintenance.  On the margin, as the economy gets tougher and people look to manage household budgets, these pets can get some scrutiny – particularly if kids have lost interest or gone off to college.  So more go up for adoption.  At the same time, requests to adopt fall, as families who might have considered an exotic pet put the “owning decision” on hold.  Taken together, these decisions cause the UEI to spike higher.  Both giving up and deciding not to own exotic pets affects Lab Store revenues “expected” in the future.  So the UEI ends up being predictive of future demand.

    Makes sense to me.

    Now, I’m a pretty good student of macroeconomics and pay attention to many economic indicators, especially predictive ones like the ECRI’s US Weekly Leading Index.  If you’re an analyst, you should too; economic indicators provide context for any analysis you might have to do, and clients often want to understand the impact of these external issues on their business.

    As far as the Lab Store specifically, I don’t usually pay much attention to the macroeconomic cycles.  The pet business tends to be insensitive to the economic cycle; people don’t stop caring for pets as the economy wobbles up and down.  That’s why it’s such a good business – if you can find a niche.  So I don’t get too concerned when I see these predictive macroeconomic indexes forecasting a slowing economy.

    However, what we have here with our Unwanted Exotic Index is a confirmation of the broader economic forecasting tools that is specific to our exotic pet business.  That makes me sit up and take notice!  Looks like our business is setting up for a repeat of the 2004 slowdown - the last time the UEI spiked like this.  Why is this important?  Because I can do something with this knowledge.  I can re-allocate and re-prioritize based on this knowledge.  For example, I can move from a “grow bigger” to a “grow smarter” mode.

    And please note: this KFI has nothing to do with traffic or sales on the web site; traffic and sales are “rear view”.  By the time you see the sales slow down it will be too late to do anything about it.  And that’s why the C-folks don’t care much about web analytics reports.  

    You could track an index like the UEI with a web analytics tool, but you’d have to come up with the idea first.  My point is you will probably have to look outside the usual “rear view” metrics to find one with forecasting ability.  I caution you not to substitute a “survey” for a predictive model; people’s opinions are a notoriously lagging indicator.  You’ll be up to your ears in the slowdown before people start turning bearish.

    So: Do you have any KFI’s – Key Forecast Indicators?  Tell us about them. 

    If you don’t have any KFI’s, now is the time to start looking for them.  What can you see now that predicts what will happen in the future?  Think about the business, think about the data sources, and put together a bunch of different ideas.  Track them back a couple of years and post them monthly going forward.  You’re bound to find something predictive.  Perhaps something about posting, like the UEI.  Recommendations / comments as a percent of visitors or something like that.

    If you’re stuck, start with a simple “engagement” idea – percent visitors / members / customers who visited / logged in / bought in the past 90 days.  If this percentage is falling, so will your business in the next 3 – 6 months.  If your business has a lot of seasonality in it, look to year-over-year comps of the same metric.

    If you’ve never played this game before, you won’t have proof your KFI’s work until after the business is in the soup, but you’ll be ready with accurate and actionable KFI’s the next time around!

    What’s the Frequency?

    Wednesday, October 31st, 2007

    The following is from the October 2007 Drilling Down Newsletter.  Got a question about Customer Measurement, Management, Valuation, Retention, Loyalty, Defection?  Just ask your question.  Also, feel free to leave a comment. 

    Want to see the answers to previous questions?  The pre-blog newsletter archives are here, “Best Article” reviews here.

    Q:  I ordered your book and have been looking at it as I have a client who wants me to do some RFM reporting for them.

    A:  Well, thanks for that!

    Q:  They are an online shoe shop who sends out cataloges via the mail as well at present.  They have order history going back to 2005 for clients and believe that by doing a RFM analysis they can work out which customers are dead and Should be dropped etc.  I understand Recency and have done this.

    A:  OK, that’s a great start…

    Q:  But on frequency there appears to be lots of conflicting information – one book I read says you should do it over a time period as an average and others do it over the entire lifecycle of a client.

    A:  You can do it either way, the ultimate answer is of course to test both ways and see which works better for this client.

    Q:  Based on the client base and that the catalogues are seasonal my client reckons a client may decide to make a purchase decision every 6 months.  My client is concerned that if I go by total purchases , some one who was  really buying lots say two years ago but now  buys nothing could appear high up the frequency compared to a newer buyer who has bought a few pairs, who would actually be a better client as they’re more Recent?  Do I make sense or am I totally wrong?

    A:  Absolutely make sense.  If you are scoring with RFM though, since the “R” is first, that means in the case above, the “newer buyer who has bought a few pairs” customer will get a higher score than the “buying lots say two years ago but now buys nothing” customer.

    So in terms of score, RFM self-adjusts for this case. The “Recent average” modification you are talking about just makes this adjustment more severe.  Other than testing whether the  “Recent average” or “Lifetime” Frequency method is better for this client, let’s think about it for a minute and see what we get.

    The Recent average Frequency approach basically enhances the Recency component of the RFM model by downgrading Frequency behavior out further in the past.  Given the model already has a strong Recency component, this “flattens” the model and makes it more of a “sure thing” – the more Recent folks get yet even higher scores.  

    What you trade off for this emphasis on more recent customers is the chance to reactivate lapsed Best customers who could purchase if approached.  In other words, the “LifeTime Frequency” version is a bit riskier, but it also has more long-term financial reward.  Follow?

    So then we think about the customer.  It sounds like the “make a purchase decision every 6 months” idea is a guess as opposed to analysis.  You could go to the database and get an answer to this question – what is the average time between purchases (Latency), say for heavy, medium, and light buyers?  That would give you some idea of a Recency  threshold for each group, where to mail customers lapsed longer than this threshold gets increasingly risky, and you could use this threshold to choose parameters for your period of time for Frequency analysis.

    Also, we have the fact these buyers are (I’m guessing) primarily online generated.  This means they probably have shorter LifeCycles than catalog-generated buyers, which would argue for downplaying Frequency that occurred before the average threshold found above and elevating Recency.

    So here is what I would do.  Given the client is already pre-disposed to the “Recent Frequency” filter on the RFM model, that this filter will generally lower financial risk, and that these buyers were online generated, go with  the filter for your scoring.

    Then, after the scoring, if you find you will in fact exclude High Frequency / non-Recent buyers, take the best of that excluded group – Highest Frequency / Most Recent – and drop them a test mailing to make sure fiddling with  the RFM model / filtering this way isn’t leaving money on the table.

    If possible, you might check this lapsed Frequent group before mailing for reasons why they stopped buying – is there a common category or manufacturer purchased, did they have service problems, etc. – to further refine list and creative.  Keep the segment small but load it up if you can, throw “the book” at them – Free shipping, etc.  

    And see what happens.  If you get minimal  response, then you know they’re dead.

    The bottom line is this: all models are general statements about behavior that benefit from being tweaked based on knowledge of the target groups.  That’s why there are so many “versions” of RFM out there – people twist and  adopt the basic model to fit known traits in the target populations, or to better fit their business model.

    Since it’s early in the game for you folks and due to the online nature of the customer generation, it’s worth being cautious.  At the same time, you want to make sure you don’t leave any knowledge (or money!) on the table.  So you drop a little test to the “Distant Frequents” that is “loaded” up / precisely targeted and if you get nothing, then you have your answer as to which version of the model is likely to work better.

    Short story: I could not convince management at Home Shopping Network that a certain customer segment they were wasting a lot of resources on – namely brand name buyers of small electronics like radar detectors – was really worth very little to the company.  So I came up with an (unapproved) test that would cost very little money but prove the point. 

    I took a small random sample of these folks and sent them a $100 coupon – no restrictions, good on anything. I kept the quantity down so if redemption was huge, I would not cause major financial damage.

    With this coupon, the population could buy any of about 50% of the items we showed on the network completely free, except for shipping and handling.

    Not one response.

    End of management discussion on value of this segment.

    If you can, drop a small test out to those Distant Frequents and see what you get.  They might surprise you…

    Good luck!

    Jim

    Speaking Schedule, WAA Projects, etc.

    Monday, October 1st, 2007

    It’s been a ruthless couple of weeks, with tons of Web Analytics Association work on top of the usual client / Lab Store stuff.  Why do the folks in the pet supply industry change packaging and labeling going into the holiday season?  That’s nuts, if you ask me, unless you think all your customers are offline stores – which I guess most of them are.  Still, there’s a large enough mail order pet business out there you would think the suppliers would catch a clue or two.  I have plenty to do during the holiday season without having to re-write copy and re-shoot photography…

    Anyway, the weeks that were.  First was a WAA Webcast on Money, Jobs and Education: How to Advance Your Career and Find Business Opportunities (site registration required, but you don’t have to be a WAA member) to get ready for and execute.

    And there was the ongoing wrestling match to establish a framework for higher educational institutions to create course offerings in Web Analytics, leveraging the course content the Web Analytics Association has developed.  Very tricky stuff dealing with these Higher Ed folks, but we think we have it figured out.  The WAA’s first partner in this area will be the University of California at Irvine – not a bad start, methinks.

    Then of course, it’s Conference season.  I’m going to be on a “Measuring Engagement” panel at WebTrends Engage October 8 -10.  The following week is of course the eMetrics Marketing Optimization Summit where I will be doing a conference presentation in the Behavioral Targeting Track and then sitting on a no holds barred “Guru Panel” with Avinash Kaushik and Bryan Eisenberg immediately after. 

    Part of getting ready for the Summit this year was a review of the WAA BaseCamp teaching materials, a pretty substantial piece of work all by itself.  We’ve done some tweaking based on comments from students in previous classes.

    Unfortunately, I have to split the Summit right after the Guru panel for the Direct Marketing Association Conference in Chicago, so if you’re going to eMetrics and you are looking to chat with me, make sure you hit me up before my presentation Tues at 1:30 PM (I will be there Sunday 10/14 @ 4 PM for the WAA meeting). 

    At the DMA, I’ll be doing a presentation with fellow web analytics blogger Alan Rimm-Kaufman in the Retention & Loyalty Marketing Track called Smart Marketing: Advanced Multichannel Acquisition and Retention Economics.  Control groups, predictive models, oh boy.

    The next day, I’ll still be in Chicago doing a real “stretch event” at the invitation of Professor Philippe Ravanas of Columbia College Chicago for The Chicago Community Trust.  Nine (9!) non-profit arts groups are battling for grant money to help execute their marketing plans, and yours truly is going to vet those plans and teach donor / membership marketing in a live format – with all nine institutions exposing their guts to me and each other -  in real time!  Budgets, response rates, web sites, direct mail, newspaper, radio, database marketing, it’s all on the table.

    Should be a real kick – if I survive the format, that is.  As a musician, I have always had a great interest in arts / donor marketing and this will be a great opportunity to interact directly with the folks in the trenches.

    So, I apologize for the lack of posts the past couple of weeks as we now join our regularly scheduled life (in progress).