Category Archives: DataBase Marketing

Lab Store: Automating Worst Practices

The news that Omniture has acquired Touch Clarity is shaking up the world of web analytics a bit.  Machine automation has always been a very sexy sell for software companies.  The problem is people think it’s a magic bullet and often end up using these tools to their disadvantage because they do not have the experience to really understand how to use the tools properly.  Then they get caught in trap of Reporting versus Analysis.

Here is a real world example from the Lab Store.  I am constantly fighting the Google AdWords A/B split testing algorithm for rotating ads.  Google almost always picks the wrong ad to run more frequently so I have to force it to run 50 / 50 in order to get accurate results.  How do I know Google is picking the wrong ad?  Because I have seen thousands of such tests, online and off, and I have a “feel” for these things based on my background in Database Marketing, Consumer Behavior and Psychology.  In each case where Google has picked one ad over another, and where I have forced it to then run the ads 50 / 50, it ends up I was right – Google picked the ad that generated the least profit per dollar of PPC spend as “best” and demoted the more profitable ad until it was not running at all.

Why does this happen?  Because Google isn’t smart enough to understand the complexity of the customer behavior in the Lab Store - and it can’t be, given the number of clients it has.  If you have done a lot of this kind of testing, you know that often the campaign with the highest response rate generates the lowest quality customers.  While these campaigns were running, I could see that the visitors generated by the campaigns Google picked as “best” were actually inferior to the visitors generated by the campaigns Google demoted, using a variety of metrics other than conversion (primarily Recency).  In other words, I was able to predict Google was doing the wrong thing by looking at the Customer LifeCycle. When I forced Google to run the ads 50 / 50 to give the demoted ads a chance, I was proven right – the campaigns Google demoted had a 90-day ROMI averaging 2.1 times higher than the campaigns Google promoted.

Look, I know these are software companies and their sole purpose in life is to create the next big thing and sell their software into it.  That’s fine, and frankly, I hope they are successful in doing it, because it will create a tremendous amount of business down the road for database marketing consultants as “machine optimization” hits the wall and companies need to be rescued from the results of it.  Just like they had to be rescued from demographic clustering in the 80’s and data mining in the 90’s.

People are always looking for the easy way out, and it ends up costing them more in the long run because they don’t really understand what the tool does and does not do. Perhaps that is simply the state of Marketing today.  So be it…

If you are an analyst and you see a black-box test result that simply does not make any sense based on your past experience, I encourage you to question the result, find a way to test it outside the system.  Learn why, because this kind of incident usually will lead to a shattering of some myth or bias you will be most happy to fully understand!

Profiling Library Customers: Update

Jim answers questions from fellow Drillers
(More questions with answers here, Work Overview here, Index of concepts here)


Another update: Robert, the questioner in this case, has pulled it off!

Read the study here (download PDF).  Very interesting conversion of RF profiles into a more visual format.

Also see dynamic visuals from the study here:

http://www.lsr-online.org/vizlib.html

and more video here:

http://vimeo.com/14374120

Indeed,  higher than expected numbers of low  recency users were found in some larger libraries but not others.

Which begs the next question, Why?

I doubt it is demographics – they get blamed for lots of things, but generally don’t control this kind of behavior across large sample sizes like this.

*Something* must be causing an effect like that – Service? Facility? Selection?

If reasons can be found, Robert is on his way to “optimizing” the library system.

Then …

————-

Q: Jim, thanks for bringing my marketing inability to a wider audence!! I’d be interested in responses you get from people doing similar work.

Just to keep you up to date on this project, we have had an amazing amount of interest from library senior management, local managers and library assistants based on my preliminary work with the data based on your methods. Interesting to see a team that is setting up / running a corporate call centre and CRM system have found out about this work and want to talk me about it – especially interesting as I work in a completely different part of the organization.

The only problem we have now is getting the data we require from our IT department. Hopefully this will be coming this month, and if you don’t mind, I’m sure I’ll be contacting you for further advice.

A: Hmm, there’s that IT thing again! Keep us in the loop, Robert!

Jim

Get the book at Booklocker.com

Find Out Specifically What is in the Book

Learn Customer Marketing Concepts and Metrics (site article list)

Download the first 9 chapters of the Drilling Down book: PDF 

Profiling Library Customers

Jim answers questions from fellow Drillers
(More questions with answers here, Work Overview here, Index of concepts here)


Q: I work for a local council in England and was recently asked to provide some local demographic profiles to help our local libraries market themselves more effectively and hopefully increase book loans.

A: Hmmm…this is definitely the first time for this question!

Q: I’m no marketer, I usually muck about with crime, economic and census data, but this seemed instinctively wrong to me after reading how Tesco’s had used its clubcard data to understand it’s market. When looking for some help, I’ve obviously found your website, bought the book, got the software.

A: I would have to agree with you on the Tesco point. If you want people to “do something”, you look at behavior. If you want people to take out more books, you look at book loan behavior.

Q: Firstly, I just wanted to thank you for making your ideas so accessible, actionable and easy to understand. I’ve picked up some other marketing textbooks for help and they seem to mainly consist of dry schematic diagrams, and bland statements. Great for a degree I’m sure, not so great for the rest of us who have three weeks to write a report on the subject!

A: Well, thanks for the kind words. That was the intent of the book – to give people the “how to do” as opposed to the “what to do”.

Q: Secondly, have you any advice or experience of this model working in the non-profit sector or specifically in libraries. For example, predicting life cycle / trigger points seems a little more complicated than the examples you use. People don’t seem to stop using the service gradually, but stop abruptly and then start up again without no warning. I’m also dealing with thousands of records. My data seems a lot less clear cut than the examples you talk about.

A: I’d agree it’s not a “clear cut” situation, but not because of the models or the channel. This kind of behavioral profiling as been used offline for decades, and it works in all kinds of situations. For example, you mention crime data, so you have probably seen that the more Recently someone has committed a crime, the more likely they are to commit another. Not that you can really take any action on that information – you can’t lock people up for being “likely to commit a crime” – but interesting just the same.

And I think you face a similar challenge.  You can run the scoring models and generally predict who is likely to slow or defect from their book loan behavior, but the question is, what do you “do” about that?  I have worked in other educational situations (likely to graduate, likely to contribute to the school after graduation) where the incentive is not straightforward but nonetheless you can create incentives to encourage people to continue their behavior.  I’m struggling a bit with how to create one in this situation, since the product itself is free.

But before we tackle that issue, I want to run through a bit of a “model” for this “business”.  It seems to me you have a market or segment shift going on.  If the primary reason people go to the library is to research a topic, clearly access to the Internet has suppressed the need for people to take books out on loan from the library.  For example, many of the trade journals that used to be hard to access or expensive to subscribe to are now available on the web.  So you have this “research” segment you have to deal with.

There no doubt is another segment, “core readers”, who simply for the love of reading visit the library to discover books and read them.  This segment is probably what I would call “good customers” because the library provides a service to them they cannot get elsewhere, the “value proposition” of the loan program matches their needs precisely.  This in contrast to the “research reader”, who now can do a lot of research from work or home on the web.

For research readers, a possible alternative would be providing access to internet terminals in the library.  But now we’re starting to encounter a different definition of “customer” and “loan a book”, right?  Let’s say, for a library, the “profit” in the venture is the “contribution to the community”, and this contribution was always measured in the past by “books out on loan”.  This metric has been the library’s KPI (Key Performance Indicator), if you will.

Let’s also say that many libraries have installed internet access terminals, as they have in the US.  Because of these terminals, you would expect that “books out on loan” would fall for the “research readers” segment, correct?  So to get a proper valuation of the contribution the library was making to the community, you would have to look at “books on loan + number of web terminal uses” to approximate the old metric “books out on loan”.

Follow?  So let’s say the real issue at hand here is the local government is trying to be “accountable” for what they spend on libraries, and they measure the “profit” of this spending by looking at books out on loan.  The library administrators are feeling some kind of pressure to “serve the community better” (increase “profit”) because books out on loan have fallen.  The problem is that “books out on loan” is no longer a viable metric – the “base” has changed, if you will.  The research segment is being served through a new method – web terminals – and this has been overlooked in terms of measuring the “contribution” the library is making.

In terms of tracking, if there is no login required to use a web terminal, someone in the library simply needs to count the number of people in a week that use the terminals x 52 weeks and use that as an approximation.  Better would be a system where in order to access a terminal, you have to enter your “library card number” or some other unique identifier tied to the person or household.

In this way, you find out more about your segments:

Researchers = only logs into computer
Core Readers = never log into computer
Multi = both logs into computer and takes books out on loan.

These “multi’s” would typically be the very best customers, since they are engaging in more than one library offering.

Above this, the library probably offers other types of services and special events.  The more of these services and events a customer engages in, the more valuable the customer is.  These are the customers the library should strive to keep active.  And like the example of books on loan, if the “value” of the library is only being evaluated on books on loan, attendance at these other services and events really should be included in the evaluation in some way.

The landscape has changed, and it’s quite possible that the metrics have not kept pace.  Perhaps it is not your place to suggest this, but as the “evaluator”, I would certainly be curious about this metric “books on loan” and make sure it accurately reflects what people think it does and is serving the administrators in the way they think it is.

Now that we have a feeling for what the background might be, you still have the issue of what do you “do” about customers who appear to be defecting?  As I said, the behavioral models will give you this information, but then what can be done with this information, especially given what are probably fairly strict budget constraints?  It’s not like you can send them a discount on their next book loan!

In situations like this, I think the best alternative might be survey work.  That is, when the behavioral models identify customers who are likely to defect or have defected, the library simply asks them about it.  For example, the library conducts a telephone survey using a sample of these people – “We noticed you have not taken a book out on loan / used a terminal in the past 3 months, is there anything we have done to offend you?  Is there a particular kind of content you are now interested in that we do not provide?  What other events or services would you like to see us provide?”  etc.

And where possible, the library should respond and provide what these customers want.  It may not be able to keep these particular customers from defecting (too late), but over time the “mix” of content and services should improve in a way that attracts and retains high value library customers.

I humbly suggest that the above approach will be far more effective and less costly than a “CRM system”.  If the library doesn’t have one, what is really needed is a “tracking system” that simply keeps track of what resources of the library each customer is using.  This will make your models much more reflective of the true economic benefits provided by the library, and give you the customer samples you need to construct effective, segment-targeted retention programs.

This kind of work also provides very good tracking for “how we’re doing” as a library and can provide excellent justifications for budgets and new requests, because it is directly tied to what customers want – no guessing games.  Over time, budget should flow to the areas more desired by customers and away from areas that are perhaps less desirable or are “pet projects” of interested parties.

Q: Once again thanks, I can at least begin to show management that we should start doing this work in Access / Excel before spending on a CRM system, but any further advice would be appreciated.

A: Well, I don’t know much about how libraries really work, but I’ve given you my best guess as to what might be going on. Hope that helps! And do keep in touch on this as you get into the guts of it, should be a very interesting project!

Jim

Update:
Robert just checked in with new info on progress on this idea, see here.