I think this is probably the last Learning and Teaching issue in Online Marketing (series starts here) before attempting to evaluate and summarize the challenge.  I would like to receive comments from you on the Sherlock Holmes Problem.

“There are two types of minds — the mathematical and what might be called the intuitive. The former arrives at its views slowly, but they are firm and rigid; the latter is endowed with greater flexibility and applies itself simultaneously to the dive.”

— Blaise Pascal

In his post How the Skills of a Night Auditor Translate into Web Analytics, Christopher Berry explores a notion we have wrestled with a lot while developing the WAA’s Certified Web Analyst Test – can you teach someone to be curious in a “business analytics” way? Or are people just born with / socialized into this skill set?  How do you measure and test someone for “analytical curiosity”?

We have referred to these issues internally on the Education Committee as the “Sherlock Holmes” problem.  The issue is not the ability to read and interpret reports, or write up findings, or anything like that.  It’s the ability to see coincidences or oddities in the data, to conceptualize linkage or relationships others don’t see, to follow the data trail (or blaze it) right down to Root Cause.

From what I have seen over the years, in both web analytics and traditional BI, there are many technical masters of the analytical tools.  Relatively few of these folks, however, also have the “Sherlock Holmes ability”.  They can answer any question you have, but they don’t come up with the questions.  That’s the Holmes side – coming up with the breakthrough questions.  And figuring out how to answer them.  The analysis, well, that’s just the mechanics.

Our friend Pascal above speaks of the “mathematical” and “intuitive” minds. I often wonder if there is something about learning “Programming” and “Marketing” that creates a rigid way of learning on either side that is mutually exclusive with the other. The intersection or “cross” of these two mindsets, the corpus callosum if you will – is seemingly rare. Can you train / learn to cross your brains? I have referred to this idea as “being a little “less scientific“ on the Technical side and a little “more specific” on the Marketing side“.

Having spent a good deal of time in the hospitality biz myself, I can tell you there are night auditors who just “run the reports” and night auditors who deliver great insights; these are the Holmes. A common “fault” you hear about these folks is “they ask too many questions”.

My wife says I ask too many questions.  Why do you want to know?

I dunno, I just want to understand it.

But why do you need to understand it?

I dunno, I just do.  I like understanding things.

Can being a night auditor teach you to become a great analyst, if you work at it hard enough?   Or were you somehow born or socialized into being a great analyst, and night auditor is just one of those analyst jobs?  There are three related “Sherlock Holmes” Learning issues in my mind: Pattern Recognition, Empathy, and Hypothesis.

Pattern Recognition – the “math” brain?

I have always described my particular skill in analytics as “I see patterns” (I know, cue the music).  I’m not sure where this skill comes from, but it’s been a skill for as long as I can remember, from taking music lessons through working on case studies in school and on into my business career.  Perhaps more importantly, I know how to translate these patterns into Marketing programs that generate increased profits (but that’s the “other brain”, see below).

Much of what Christopher talks about in his post is pattern recognition, or often more importantly, the deviation from the patterns you expect to see.  For example,  (Hypothesis) closing the kitchen at 11 PM will increase total liquor sales across the dining room and bar.  Once you analyze and prove this, it’s easy for others to analyze and follow (and post on blogs and have conferences about!)

But how about the ability to even conceive this question in the first place?  Not to mention running through the possible outcomes – what if they drink less $100 a bottle wine in the dining room and instead have $5 drinks in the bar?  How many $5 drinks would they have to have   Is my projection logical?

I know, depends on the audience…

But seriously, how do you teach people Pattern Recognition, or to Recognize Lack of Pattern?  How do you test people for this capability?  Rorschach Tests?

Empathy – the “marketing brain”

I’ve seen a lot of material on analytical thought – right brain versus left brain mix, deductive versus inductive thinker, etc.  Sometimes I think analytical ability has a lot to do with Empathy – can you understand and visualize the players, think about what they would do.  How many “boozers” are there in the dining room on a Saturday night, and what would they do if we closed the kitchen at 11 PM?

Perhaps this pull towards Empathy as a key component of being Sherlock Holmes is my Marketing background speaking, because it’s very common in Marketing to have to “step into the shoes” of the customer or audience.  Again, I can tell you from experience, it is difficult for many analysts to “Pretend you are a woman over 50”. Even more difficult – and often more critical – is to then answer the question “Given you are a woman over 50, what would you do or think?”  The concept of Personas is meant to address some of this problem, make the challenge more tangible for people.  But you still have the “what would the Persona do or think” issue.

How do you Teach or Learn this skill?  I think there is a concrete answer here – you simply have to read a lot of valid research (not this kind), and then take the time to think about it, connect it to experience.  I do see the tremendous amount of twisted and corrupt research that gets passed around the blogosphere as “truth” being a severe Teaching / Learning challenge though.  No wonder people are confused and always searching for “the truth”.

Hypothesis – the brains meet

Once you know using Landing Pages is a good idea, it’s not very hard to measure and report on that idea, replicate and copy it, spread it through conferences and blogs.  It’s an important idea, so important in fact we are still talking about it 10 years later.

But somebody had to invent the concept of a Landing Pages in the first place.  And when that idea came out of seemingly thin air, also devise a way to prove it was a good idea.  Hypothesis and Test.  It seems to me Online Marketing / Web Analytics is absolutely chock full of Tests, many of them without a Hypothesis.  Instead, we have a lot of brute force Testing, and then a winner is chosen.

This could be why there is a glacially slow learning curve; in fact, all this testing is “one-off” activity and people really are not Learning the fundamentals, not connecting the dots, as it were.  So these folks just keep Repeating the Past. Testing without a Hypothesis is a very robotic way to look at Online Marketing and Analysis, and at the very least wastes a lot of time.  This brain “Arrives at its views slowly, but they are firm and rigid”, as Pascal would say.

Do people really have to repeat the “Landing Page” test with every new piece of online media?  I guess you would have to, if you didn’t have the Hypothesis “message continuity in a sequence of online advertising pieces will result in a higher response rate”.  I can tell you it does.  I can tell you it always has offline as well.  It’s just a fundamental rule, especially in direct marketing, for example:

Outer Envelope > Teaser > Open Envelope > Johnson Box > P.S. > Letter > Reply Device

The better integrated these messages are – the stronger the “scent” – the higher the response.  There is not a need to keep testing this idea.  Think about it – does it make any sense to you, is it possible that a random series of disconnected messages would get a higher response rate?  Really?

Lack of  Hypothesis for each test may be the most crippling Learning problem out there, and perhaps the most difficult to crack because it requires both brains to be in gear at the same time. In other words, creating a breakthrough Hypothesis probably requires both Pattern Recognition and Empathy.

In case the concept of Hypothesis is not quite clear, I don’t view an “educated guess” as a Hypothesis; you need to provide concrete reasons for the Hypothesis, fully explaining why the outcome should occur.  The good news underlying the Hypothesis approach is this: by being proved wrong or right every time – instead of just waiting for outcome – folks will start to Learn.  Just shrugging your shoulders and accepting the outcome of a test doesn’t create Learning.

Write it down, what do you think the result of the test will be and why?  At some point, people will begin to say to themselves, “Hey, this test looks like these other ones.  My Hypothesis is…”

And more often that not, they will be correct.  Sometimes they will be spectacularly wrong, and that’s OK, because Failure is a Learning Experience.

Questions

So that’s the Sherlock Holmes Problem.

Comments on any of the above?  Are great Analysts born, socialized, taught?  Have I completely missed the core issue, and something else – lack of proper analytical culture, for example – is really the root cause of a slow learning curve?  HIPPOS don’t know how to form a Hypothesis, and resent one being formed for them?

Anyone know of resources (preferably focused on education) dissecting these issues / providing learning solutions?