Category Archives: Analytical Culture

Inside WAA Certification: Any Questions?

Analytical Culture, Web Analytics

The WAA has published a lot of info about the new WAA Certification Exam; you might want to first read the FAQ and take a look at the application information and Exam Handbook for the organizational details, and you can see sample questions from the Test at the bottom of the page here.  But something I can just about guarantee about the Certification – no matter how much info the WAA publishes about it, many people will still have questions!

So here, I will attempt to answer other kinds of questions I think people might have based on my discussions with WAA members.

Update: The WAA has answered many Certification questions here.

However, I’m going to approach this topic a bit differently than most of the published documentation – from a Product / Marketing perspective, rather than an Educational / WAA POV.  I can do this because (if you don’t know) I have worn all the hats on this project – developer, marketer, WAA project owner – and I think it might be helpful to tell the business story of the WAA Certification, from the bottom up.

And if you have other questions, feel free to leave them in Comments and I will do my best to answer them!

Continue reading Inside WAA Certification: Any Questions? →

Tortured Data – and Analysts

Analytical Culture, Web Analytics

Fear and Loathing in WA

You may recall I wrote last year about the explicit or implicit pressure put on Analysts to “torture the data” into analysis with a favorable outcome.  In a piece called Analyze, Not Justify, I described how by my count, about 50% or so of the analysts in a large conference room admitted to receiving this kind of pressure at one time or another.

Since then, I have been on somewhat of a personal mission to try to unearth more about this situation.  And it seems like the problem is getting worse, not better.

I have a theory about why this situation might be worsening.

Companies that were early to adopt web analytics were likely to already have a proper analytical culture.  You can’t put pressure on an analyst to torture data  in a company with this kind of culture – the analyst simply will not sit still for it.  The incident will be reported to senior management, and the source of “pressure” fired.  That’s all there is to it.

However, what we could be seeing now is this: as #measure adoption expands, we find the tools in more companies lacking a proper analytical culture, so the incidents of pressure to torture begin to expand.  And not just pressure to torture, but pressure to conceal, as I heard from several web analysts recently.

Continue reading Tortured Data – and Analysts →

Control Groups in Small Populations

Analytical Culture, DataBase Marketing, Driller Q & A, Newsletters, Web Analytics

Jim answers questions from fellow Drillers
(More questions with answers here, Work Overview here, Index of concepts here)

Q: Thank you for your recent article about Control Groups.  Our organization launched an online distance learning program this past August, and I’ve just completed some student behavior analysis for this past semester.

Using weekly RF-Scores based on Recently and Frequently they’ve logged in to courses within the previous three weeks, I’m able to assess their “Risk Level”– how likely they are to stop using the program.  We had a percentage who discontinued the program, but in retrospect, their login behavior and changes in their login behavior gave strong indication they were having trouble before they completely stopped using it.

A: Fantastic!  I have spoken with numerous online educators about this application of Recency – Frequency modeling, as well online research subscriptions, a similar behavioral model.  All reported great results predicting student / subscriber defection rates.

Q: I’m preparing to propose a program for the upcoming semester where we contact students by email and / or phone when their login behavior gives indication that they’re having trouble.  My hope is that by proactively contacting these students, we can resolve issues or provide assistance before things escalate to the point they defect completely.

A: Absolutely, the yield (% students / revenue retained) on a project like this should be excellent.  Plus, you will end up learning a lot about “why”, which will lead to better executions of the “potential dropout” program the more you test it.

Continue reading Control Groups in Small Populations →