I’ve just scrapped a long and detailed post on the topic of how survey results are reported and used, particularly in the promotion of online marketing topics. But I’m thinking maybe everybody else is hip to this topic, and I’m just old.Â
Seems like a lot of folks use “surveys” specifically to generate press, it’s like a formula now. When you look at any of the methodology – when it is rarely exposed – it often looks like crap.
I mean really, an online survey of a bunch of your customers or newsletter subscribers is fine, but then it gets reported like “research”. If it’s research, how come there is never a “margin of error” reported? A comparison of the folks who answered versus the folks who did not, of the potential bias in the sample?
Jim,
Interesting post. How come you scrapped the longer post? I would venture that few people are actually “hip to the topic” but I have to agree with your assessment of poor methodology statements. This is why Zori and I include a methodology in our research (which I kind of doubt anyone reads, but it’s there, and Zori’s email address is in the work as well for people who have questions about the methodology, etc.)
Anyway, I think we’d all benefit from your thoughts on the subject. Oh, unless you’re going to use me as a strawman to beat to death, in which case I love the short, non-specific post. ;-)
Eric T. Peterson
http://www.webanalyticsdemystified.com
I scrapped it because I’m not sure people care, it seems to be taken for granted lately that “survey = truth”, so I’m just testing the waters to see if people are interested in the topic.
I wasn’t thinking of web analytics in this regard, where I assume people have a bit of rigor; as you pointed out, you publish background.
More so, I was thinking about the river of social media-related stuff that is pounded out day after day, with what seems to be little rigor in the methodology and damn little fact-checking by “reporters”, even when the facts are right there available to check…
So that’s why I wonder if people care at all about this kind of topic…maybe it’s just a product of being able to say whatever you want whenever you want, then it gets picked up and syndicated all over the world, and then it becomes a fact.Â
But Hype is one thing, that’s OK with me. Outright distortion or torturing of the survey data is quite another, in my opinion.
So, have people already caught on to the fact a lot of this survey stuff is being classified as puffery, thus evading any legal issues about “facts”?
If so, topic not worth discussin’…
Yeah, I hear ya. But I don’t see why it’s not a topic worth discussing. I think people make assumptions all the time without thinking about A) why they’re assuming and B) the ramifications of said assumptions.
Don’t get me wrong, I’ll torture data all night long if need be, but I’d love to read your insights. That said, don’t write it for me … write it for you.
Your avid fan,
E.
OK, see:
Research for Press Release
Â