Category Archives: Newsletters

Tracking UnTrackable Campaigns

Jim answers questions from fellow Drillers
(More questions with answers here, Work Overview here, Index of concepts here)


(Any Tracking is Better than No Tracking at All)

Q: I have a background in direct marketing and the measurement of campaigns using defined sources.  Now I am working at a technical or 2 year college and I’m trying to use my previous knowledge and experience to measure our return on marketing efforts in recruiting new students and converting them from prospects to enrolled students at the school.

I’m looking to measure events and all of the advertising and promotion used to communicate those events as well as other campaigns. It’s difficult to measure since it may include newspaper ads, direct mail, posters, etc. – not just direct mail with a measurable list to refer to for tracking response.

Plus, trying to get the admissions folks to track leads can be difficult. They also want to track return on publications (brochures, flyers, etc.)….not sure how one would do that if they are not mailed to a given measurable list?

I’m looking to track ROI based on the whole equation….from the # of inquiries who came to an event or responded to whatever all the way to whether they matriculated and enrolled in courses.

A: I can feel your pain!

I don’t think there are any easy answers to this. You could simply measure what you can measure through the traditional direct methods you are familiar with, and let the rest “ride”. Or, you can try to selectively determine, as best you can, what the value of all these other activities is by engaging in some kind of testing. This will take some institutional willpower and is probably something you can’t do on your own. In other words, I encourage you to start evangelizing the next generation of marketing measurement at the school.

From your title, I perceive you don’t “report to” marketing, but some higher institutional level responsible for Quality and perhaps Productivity / Accountability for funds that are spent? It sounds to me like your unit might perhaps report into the Financial area of the school at a higher level, and if so, that’s good!

I think I would simply start the conversation with folks in the financial area about some of these issues, and see if you can create some simple tests to get some “direction” on the contribution of the various marketing outlets.

For example, every publication should contain some kind of tracking device. Sometimes you have to be creative with this idea and it won’t always be accurate, but it’s better than nothing at all. If response is generally by phone, then try to get a unique phone number for newspaper ads, brochures, etc. If response is driven to the web site, get unique URL’s put on each document. If response is filling out a sheet or card at an event, have them numbered or coded in some way. Then of course, you need to get the response information – number of phone calls to each number, number of visits to each unique web site URL, number of response cards mailed or turned in.

Implementing a program like this, then finding and getting access to the response info may not be easy, and that’s why it would help to have a “higher power”, particularly a financial one, backing this effort. It’s pretty amazing what people will do when, for example, the people who control the budget for an area say, “You will participate in this tracking program”.

Alternatively, you could go with a test / control kind of scenario where during a quarter, you leave out one particular marketing effort and see if there is an impact on overall Marketing Productivity. This is more of a “marketing mix” kind of approach and not without some problems, including proving the missing marketing effort was responsible or not for changes in Productivity. You have to think about how you might pin these issues down in advance – for example, do you have good baselines for “normal” activity?

Whether or not you decide to pursue all of this is somewhat of a personal choice. Some analysts simply don’t think it is their “job” to help create measurable structures – they only measure what can be measured. Others see the difficulty as a challenge, and want to help build out the structure. Clearly, if you are going to eventually be responsible for measurement, being a part of the team constructing the measurement paths is a real advantage to you. It will involve some politics, but analytics always involves politics at some level. I encourage you to seek out the support you need to make this work.

If there is “pressure” for measurement, someone wants it to happen. Start by finding these people and having a conversation about how it could happen, the strengths and weaknesses of the measurements, the internal challenges you will face.

When taking on something like this, it’s usually best not to try to change the world all at once, but one step at a time. So, for example, looking at the overall “unmeasurable budget”, what is the largest line item? If it’s “newspaper”, that’s a place you probably have the largest leverage. Implement there first, keeping in mind that this single implementation might help you down the road. For example, getting a unique phone number and results tracking for newspaper may teach you a lot about how to get this done for other marketing devices.

The finance people should be able to provide you with some idea of the net “margin” of a course and any other financial ideas that come into play. Then it’s a matter of asking if the spend on the media generated positive results. If the margin on a course is $500, a newspaper ad costing $1000 that only generates 1 student is not a great investment – but it might be the best one relative to other vehicles. This part is not really your call. Your job is to bring the data to life so that people can understand what they are spending and what they are getting.

There could be plenty of reasons why “losing $500” on a newspaper ad is OK – there is “brand exposure”, for example. In this case, the brand exposure only costs $500 versus a perception that it costs $1000, if the student generated is included in the formula. This may be a very positive result of the measurement for many folks in the institution. That judgment is really for someone else to make. Now at least they are making it on a full set of facts as opposed to perceptions they have about cost.

Q: However, how long do you keep measuring enrollments…they may not enroll based on one campaign…might take a few hits before they actually become students.

A: Sure. What seems reasonable? Given an annual budget cycle, let’s say reasonable is 12 months. One benefit from your tracking is you will be able to probably put some numbers against this eventually. If you get calls to a brochure number 2 years after it was issued, then the number is 2 years for a brochure. Newspaper calls stop coming in at 4 weeks, it’s 4 weeks for newspaper.

Q: The other issue: the marketing folks only want to measure up to inquiries–what they have control over.  What’s the best way to only measure the return on that…it’s before a “sale” or “enrollment” even occurs, so the “profit” is not booked.

A: Well, sometimes you simply have to decide what is “best available”. You certainly can start by measuring inquiries, especially since it’s pretty clear in this case marketing lacks some control over key conversion elements – financial aid, student abilities, and so forth. Down the road, it’s possible that certain types of media generate lower quality inquiries with lower conversion rates. You will get there over time.

For now, you could apply the “average conversion” to any lead to get down to the financial part of the game. If all leads on average convert at 25%, then just use that.  Then when tracking gets a wider reach, try to drill into it more deeply. To do this, you’d have to get access to enrollment data, of course. But you don’t have to get “all the data, all the time”. You could do a sample of a couple of months and go through it by hand to match back to inquiries, if you have to. You certainly would not be the first to do something like this to pin down an issue – it happens all the time.

Whether you want to do something like a “by hand count” or see it as part of your job is really more of a personal choice.  You can certainly – and analysts often do – blame a lack of knowledge on system problems, politics, whatever.  Just can’t get the data.  For some people “don’t know” is not acceptable – they have to find the answer, whatever way they have to do it – even if it is by hand!

Q: Does your book give an education example? We’re not “selling” a product and “sales” deals with # of credits taken by students & price per credit plus funding we receive from the state gov’t based on the number of FTEs generated. It seems like such a different animal so I’m struggling to figure how to do ROI for an educational services provider. I have created an Excel template based on how I calculated ROI for other industries but haven’t tried it yet.

A: Sounds like the “profit” side of it is a bit complex with the outside funding, but I’m sure you can get to a “value per FTE” somehow. Just start with something, and make it better as you move along. If you have to, simply take “all revenue” divided by number of FTE’s and you at least have a place to start.

The book doesn’t have extensive educational examples but here are some related topics from the newsletter:

Predicting Student Churn

Profiling Library Customers

I have a lot of interest in these educational measurement scenarios for a variety of reasons.  Keep me posted on how you are getting along and ask any questions you might have as you make your way through?  If I can be of help let me know!

Jim

Get the book at Booklocker.com

Find Out Specifically What is in the Book

Learn Customer Marketing Concepts and Metrics (site article list)

Download the first 9 chapters of the Drilling Down book: PDF 

6033% ROI, Defining Churn

Jim answers questions from fellow Drillers
(More questions with answers here, Work Overview here, Index of concepts here)


6033% ROI
—————-

Q: We have exchanged email a few times, and I don’t recall if I ever said thank you for your book.  While I had been experimenting with many CRM programs in my little dry cleaning shop, your book gave my thoughts order and clarity to refine what I had started.  Today, I see the world differently.

A: Well, thanks for the thanks!

Q: You may or may not remember me. Just after I sold my dry cleaning shop, I had bought your Drilling Down book. I was the dry cleaner who had been doing rudimentary data mining and CRM with a point of sale system I had developed in Regina, Saskatchewan.

A: I do remember. Internally, I was thinking, “Wow, this is going to be a real test of the Drilling Down concept”? I mean, I have seen it work in many small businesses, but dry clean (seems to me) is a very tough, tough business. Too many players, a lot of competing on price, etc. A great environment for underground customer marketing in terms of beating the other guy – they will never know what happened to them. But still, tough for small owner / operator to have the “will” and time to really make it happen. So yea, I remember…

Q: Well, I’ve continued working within the dry cleaning as a marketing consultant. The programs I had developed in my shop have now been transplanted into a few of my client’s shops, and are bearing fruit.

Tonight one of my clients reported a ROI of 6033% doing direct mail to certain customers in his market in California. Another client of mine reported his fourth year of steady growth. One of my first clients has been showing a 7 percent annual compound growth, and he is in a flat or declining market. What began in my shop has been proven across North America, into Europe and Australia by my clients.

A: I can’t express how exciting that is. Congratulations!

Q: Jim, data mining dry cleaner’s data is a blast. You would be stunned at the quantity, and quality of data a dry cleaner gathers today. Would you ever have thought data mining could be applied to suits and shirts? Well yes, it can.

A: I am stunned, and I bow to your most excellent Drilling!

Q: Once again, thank you.

A: And thank you for sharing this, it’s very, very exciting to hear. Like you said, no other word for it than “stunning”. I remain most stunned!  Keep me informed. Perhaps you should write a book?

Jim

Defining Churn
———————

Q: I work for an economics consulting firm based in Washington DC. I am researching customer churn and customer displacement statistics across a variety of industries to try to establish a benchmark of what is considered high and low customer displacement.

A: Nice to meet you, and a noble task!

Q: Do you happen to have any such churn statistics, or know if a place you could recommend?  I found plenty of statistics regarding churn rates within the telecom industry, but am most interested in companies that are involved in business-to-business relationships with their customers (relationship between a customer and a supplier).

In addition, I would also like to find churn statistics for customers who use multiple suppliers. For example, a customer may go to several grocery stores rather than sticking with one dedicated store.  I would be interested in learning more about the statistics companies in these types of industries use to track customer displacement.

A: The reason you find a lot of churn info in telco / cable is the end of the customer life is easily defined by the disconnect, and these numbers are reported publicly as part of annual reports and so forth. In many other businesses like the ones you describe, typically the companies have failed to define customer defection and so in their minds, there is no churn because there is no defection.

A “customer”, even though they have not contacted the company for 3, 5 or 10 years, is always still a customer. If the company thinks like this there is no churn rate to be measured, by the definition the company has chosen for itself.

At the same time, defining defection is pretty easy to do by looking at the transactional data and defining the patterns of defection, for example “if a customer has not ordered from us in 3 years they are highly unlikely to order again”. That’s defection defined; you just put a line in the sand and say “3 years no contact is a defection”. The company then should declare customers in this status “defected” and then a churn rate could be found. This is pretty easy to do, so if not executed, one of two situations exist: either the company does not have the data or they don’t have the “will” to discuss, internally or externally, the concept of customer defection.

A third possibility exists: the company in fact has the data and has defined defection, but would never, ever speak to churn or customer defection in any kind of public forum because this information is so critically important from a competitive and strategy perspective. To discuss these numbers or the implications in public could have dramatic consequences for company positioning in the market or stock price. So if they have the numbers, they’re locked in a safe.

As a result, I’m sorry to say, I do not have any broad-based “sources” for you, save one possibility: a book called The Loyalty Effect by Frederick F. Reichheld (1996). In this book, Reichheld goes through the business models of 25 different companies that excel at retaining customers in different industries , and proves out the financial model of customer retention using real data. This is the book where the quote, “It costs 5x more to acquire a new customer than retain a current customer” (or the various bastardizations) came from. So it might help you out.

The only other thing I can suggest is that “churn” is not always the word used to describe these stats but is most often used when the disconnect is easily defined, as in telco / cable; “displacement” is a rare use for this idea as far as I can tell..

“Customer Turnover” is a popular phrase in Europe and is used by some in the US; also “defection rate” is used quite a bit. So if you’re pounding on Google to try to find these numbers, try those phrases and others you may find when doing these searches. Banking / finance / insurance is another area where the “disconnect” is often easily defined, so you will find various defection rates in some of their case studies on the web.

Jim

Get the book at Booklocker.com

Find Out Specifically What is in the Book

Learn Customer Marketing Concepts and Metrics (site article list)

Download the first 9 chapters of the Drilling Down book: PDF 

Optimizing Mail Drops for Consumables

Jim answers questions from fellow Drillers
(More questions with answers here, Work Overview here, Index of concepts here)


Please note: The business discussed below is a “continuity business”, where customers consume the product and need to either reorder from the company every few months or seek alternatives sources for the product. In this scenario, the behavior of customers is generally governed by the Latency Metric.

Q: Currently we mail our current customers direct mail every 6.5 weeks.  We have a new VP and he is asking if that is the optimal spacing of mailings.  I’m wondering if there are any best practices for setting up frequency tests?  If you can shed any light on how to set up such a test I would greatly appreciate it.

A: Well, do you know how you got to the 6.5 weeks number in the first place?  Somebody must have thought it was a good idea based on some kind of data (I hope)!

Obviously, there is some significant financial risk in simply “moving the drop around” and testing results that way.  You can do it, often by slivering off parts of the drop and dropping then at different times, but there could be a substantial financial penalty for approaching the problem this way – both on the cost and sales sides.  This is especially true when you have a current schedule that seems to be working.

The first thing I would do, if possible, before taking on the risk of messing with the mailing is to see if you can find any segmentation /  frequency that makes more sense from the customer data itself.  Since you also have a web site, there probably is evidence of “natural purchase cycles” the customer engages in that operate outside the mail drop – customers ordering “when and how they want to”.

Can you find evidence that the average purchase cycle is more like 5 weeks or 7 weeks?  How does this differ by product line, or packaging of the product?  Both segmentation by actual customer behavior and segmentation by product line will generally provide increased profits, provided the cost of dropping different mail streams does not overpower the increased sales.

For example, if someone can buy a “90-day supply”, well, 6.5 weeks is a bit  early for the mailing, I’d think.  If they can only buy a 30-day supply, well, it seems to me that 6.5 weeks could be a bit late.  Look to actual  purchase cycles by product line / supply length and see if you can find any patterns in the purchase behavior.

The key to this kind of analysis is to line up all the customers so that the purchase cycles match.  In other words, you need to enforce the same start date.  One way to do this, for example, is look at all new customers who started in January 2007; of the ones that bought again, when did they purchase – 5 weeks, 6 weeks, 7 weeks out?  What percentage of new starts in January (or any other month) purchased in each of the subsequent weeks?  Be aware choosing a single month may create results that have a seasonal bias, but I’m not sure that is relevant in a product line like yours.

A more complex but possibly more accurate way to do this is to “normalize” the start date of all new customers in 2006 and then look at the subsequent purchase patterns – given the same start date, what percent bought again 5 weeks out, 6 weeks out, 7 weeks out? You can achieve virtually the same thing by taking each month of current year and running it through the same drill as the one described above for following year, though it won’t be as accurate.

Once you have nailed the cycle for new customers, you can move on to see if  there is any change in optimal cycle date as customers age.  My guess is the cycle probably gradually lengthens until the customer defects.  If this is  true, it might be worth it to do two mailings with different cycles – one cycle for customers who became new customers in the past (say) 6 months and all other customers.  It’s likely in this business there could be an important behavioral difference between new and current customers that would allow you to deliver a more optimized mailing cycle.

Failing access to any analytical means to drill down into the data first, because either you lack the resources or simply don’t have the time, set up your next drop with flagged segments based on “weeks since last purchase” and look at profit per customer.  You could also back into this if you have good promotional history on your customers.

In other words, if you are going to drop “everybody” at the same time, there must be a segment where for this single drop, the time since last purchase based on arrival of the mail is 5 weeks ago, 6 weeks ago, 7 weeks ago, and so forth.  If you flag these segments before the drop in the database, you should be able to go back and determine sales per customer mailed for each segment.  This will tell you if your timing should be adjusted.  Further, you might divide these time-based segments, if there are enough members in the  segment, along various product lines.

Then, once you have a handle on the general cyclicality of different segments, you can get to profit per segment by using control groups to measure the lift and profit by segment.

A careful analysis of the next drop (or as I said, a previous drop if you have good history) should tell you which drop cycle for each product line is optimal.  From there, you have to look at economies of scale and decide if  you can afford that kind of segmentation.  You may find that due to the economies of scale in the mailing, you simply cannot drop 50% of your mail one week and the other 50% the next, for example.  But you might find enough support in your analysis to either justify the current 6.5 week drop as the most efficient, or to move it up or back somewhat.

Another way to approach the “timing problem” relative to economies of scale would be to try “reminder to re-order postcards” instead of mail or catalogs to some members of the group that require special timing considerations.  For example, new customers might not really need a catalog on their first drop, a postcard driving them to the phone or web site to reorder might be enough.

No silver bullets, I’m afraid. Just good ‘ol fashioned sloggin’ through the data ought to get you to where you want to go!

Jim

Get the book at Booklocker.com

Find Out Specifically What is in the Book

Learn Customer Marketing Concepts and Metrics (site article list)

Download the first 9 chapters of the Drilling Down book: PDF