Monthly Archives: December 2006

Root Cause: The Check Shredding Example

Most remote retailers have problems with processing checks, or more accurately, the lack of processing them.  People place orders using “check” or “money order” as method of payment – then simply fail to send in the payment.  This of course creates all kind of waste in the system, from order taking and customer service through inventory management, stockout and available for sale  “Check fallout”, as we called it at HSN, was a problem that if it could be minimized, would drive cost savings in almost every silo of the company, in addition to increasing revenue.

It was my job to “fix” check fallout.  Yea, not exactly a “traditional” marketing thing, but I was game for trying it, because I knew it impacted the overall financial success of all my marketing programs.

First, of course, the analysis.  It ends up (surprise, surprise) that 90% of the check fallout was generated by 10% of check buyers.  These “sport buyers” as we called them simply were placing check orders to fulfill an interactive shopping need of some kind with no intention to ever send in the check – much the same way as people “Add to Cart” with no intention of checking out.  This behavior is non-controllable and so from a marketing perspective, there is really nothing marketing can do – but that doesn’t mean we ignore it.

It was time to call in the cross-functional Business SWAT team.

The business SWAT team heads over to customer service and IT to explore the potential benefits / downsides of creating a “threshold” program of some kind which would prevent sport buyers from placing too many check orders.  We’d simply base it on history: when a customer hit (for example) 60% of check orders falling out, we would prevent them from placing a check order.  When we’ve built the idea, it’s off to Finance with an impact model – sure, we may lose some sales, and sure, we may generate some additional customer service calls, but the savings in order taking time / inventory turns / sales opportunity costs outweigh the downside and the program is approved.  IT builds it and creates a monitoring facility / reporting tic in rep screens so we could keep track of calls generated due to the implementation.

OK, so what about the controllable fallout?  We built a regression model that did a pretty good job of predicting who might be prompted to send in the check and created a marketing package for testing. 

Essentially, a lot of these controllables switched back and forth between credit card and check payments, so the thesis was they were simply “forgetting” and the marketing package would be a “reminder”.  Make it easy to send in the check by providing an “invoice” of sorts and a pre-addressed envelope.  Since the target group were generally good customers, position the package as an “additional service for our good customers” kind of thing.  The tests went well on the ROI side (random sample versus a control group) and so we rolled it out.

Rollout did not go well on the ROI side; after costs, control was more profitable than the test group.  So we start with the 5 Why’s, and (as we often did) went directly to the customer for some answers, starting with those that had a very high predicted response but did not respond (send in the check).  The majority of these customers told us they did send in the check, and “appreciated the reminder”. 

OK, so the copy works (customer finds the “service” a good idea), but what’s up with these checks, is the customer lying about sending them?  Possible, but the consistency of the “appreciated the reminder” response doesn’t really head you in that direction.

So the business SWAT team played what became one of our most popular problem-solving games: “be the problem”.  In this case, “be the check”. This game involves following the entire process for a check through every potential place it could go and be touched, no matter how small, no matter what silo.  Mailroom, everything.  This leads to the discovery, in a remote area of customer service, of the “check shredding group”. These people are tasked with shredding checks from customers every day – checks these customers sent in to pay for orders

Talk about your 5 Why’s.  Why are the checks shredded? 

Because the customer did not sign the check.

Um, can we hold it right there?  I’m not sure I need any more Why’s, but I do have “1 WTF”: Where is the business rule that says “customer did not sign check, we should shred it”? 

And a few clarifiers: Do we notify the customer that we shredded their check (of course not).  Do we post anything to customer service screens to provide a “status” on the order of “shredded check” so agents know what is going on?  (Of course not).  Who owns this rule?  (Finance).

So it’s off to Finance.  How do we get them to play?  “Um, guys and gals, we have about $500K in check payments for orders we are shredding every week and the people doing the shredding say they’re doing it because you told them to.  Could we review this little ‘ol business rule with you folks?

First, is it against the law to deposit an unsigned check? (No, not if it was sent to you to pay for an order). So there is some other kind of risk, a financial one? (Yes, what if the customer disputes the order, they can say they didn’t sign the check, and we’d probably be out the merchandise due to “unsolicited merchandise” law).  OK, let’s say that would even come into play, what is the financial risk relative to the potential gain?

Given the target group are some of our best customers, and given the customer service problems (Where’s my package?  I sent you a check weeks ago!) this causes, do we have a feeling for the real financial risk versus the benefits of simply depositing the checks and shipping the merch?

This is, of course, an answer a mid-level Finance person is not going to give on the spot, so he gets it on the agenda with Finance, which then needs to go to Legal.

But the word comes back down the line a week or so later: stop shredding the unsigned checks.

And as if by magic, all of a sudden the check fallout marketing program we designed starts pulling in a 30-Day ROMI of 58% (measured versus control group).

Lessons:

The success of Marketing programs and Marketing in general is often determined by factors outside the control of “Marketing”.  This means the highest ROI customer marketing projects are typically cross-functional in nature, because they usually represent process problems that have been ignored due to “not my job”-ism.  Any Marketer who wants to step up to the plate and get a seat at the strategic table should analyze these problems and work out solutions.

Cross-functional SWAT teams that are “pre-built” and ready for action are a tremendous asset to a company.  Usually made up of middle-level managers, these people have deep connections into their departments and know where the answers are.  Most importantly, they do no seek “to blame” but “to fix” and this is well understood in a company with a healthy analytical culture.  Managers do not fear the arrival of the business SWAT team at their office door, they welcome it.

Homework for Marketers who want to be more Productive:

Do you know the business rules surrounding payment processing at your company?  These rules can affect the success of your marketing programs.

Root Cause: The Five Why’s

There are many marketing productivity situations you will encounter where a change in the metrics, positive or negative, is being caused by something you or your area has no control over. For example, your marketing campaign generated a lot of leads but these leads failed to convert at an acceptable rate. Since this campaign has always generated good leads in the past, you ask yourself, “What happened? What was different this time?”

Upon further research, you find that the leads were of the same quality, but for some reason most of the leads failed to make it to the salespeople they were intended for, and instead went to a brand new salesperson in training. This “process failure” resulted in a low conversion rate, and is the root cause of the campaign failure.  Here is the head’s up on root cause: make sure when analyzing campaigns and other metrics that you understand what “failure” you are really dealing with. Relentlessly search for root cause.

Getting to the Root of It All

I often speak about cross-functional teams in the management of the analytical culture, and the concept of root cause is the driving reason for this, as the road to root cause is often cross-functional in nature. Having a cross-functional mentality in place ensures that these complex root cause issues are addressed without finger-pointing. In a good cross-functional team, there is no “blame”, only learning and continuous improvement. If you don’t chase down the root cause of your issues, the problem likely will continue, creating pain for all members of the team. 

We can look to the practice of Six Sigma for some help in performing Root Cause Analysis. A popular technique is known as “5 Why’s”. The “surface” problem or “effect” as it is called in Six Sigma is written down, specifically, at the top of the page. Then the team asks, “Why did this happen?” This answer, or “cause” in the language of Six Sigma, is written down underneath the effect. “But what caused this to happen?” is asked, answered and written down under the previous cause, and the process repeats until the team agrees that the “real”, or “root cause” of the problem has been arrived at. 

If you solve this last problem, you will fix the root cause, which typically means you will solve all the other “causes” above it.

For some reason, the root cause is often reached around the 5th “Why”, hence the name. You may reach your root cause sooner or later depending on how well the problem is understood and defined in the first place.

Let’s take the example of the lead generation campaign above and run it through the 5 Why’s process:

Effect: The campaign failed to generate leads that converted at an acceptable rate.

Why?

Cause: Because the leads were distributed to a sales person who was in training

Why?

Cause: Because the sales manager who usually distributes the leads was out of town and the only person available to distribute leads did not know the leads are not supposed to go to sales people in training

Why?

Cause: Because there are no rules in place to guide somebody on how to distribute leads when the sales manager is not available

Why?

Root Cause: Because the sales manager has not provided specific instructions on how to distribute leads

Solution to problem of under-performing campaign: 

Have sales manager create lead distribution rules and appoint a person to control distribution of leads in her absence. Probably a good idea to appoint someone to control distribution of leads if both the sales manager and the 1st level appointee are unavailable as well, so any “cascade” effect is stopped and campaign budget not wasted.

See how that works? The root cause doesn’t have much to do with web analytics or marketing, but the impact is felt there and as such, the cross-functional root cause for failure should be uncovered and repaired.

Now, as an analyst, you may want to “tweak” the “5 Why’s” process a bit. While useful in getting a discovery process going, particularly in cross-functional team settings, there is a real problem for the analyst. Do you see it?  

Here it is: where is the analytical rigor in this process?  For each cause the team agrees to, I would ask for proof or evidence of the causal relationship, or else you might end up going down a dead end road.  Examine this proof and determine if it completely explains the relationship between the effect and the cause.

***** Mining for Creative Ideas

“Doing something” with the data seems to be the place everybody gets stuck.  They can run reports, create segments and so forth, but then get bogged down with what to do next.  This article talks about how the behavior you see in the database can be converted to creative execution for both B2B and B2C.  Five stars for talking about the difficult interface between marketing and technology people and providing concrete suggestions on how to deal with it.