Thanks

K

The answer to yur question is: perhaps but not really. This is a bit complex and I’m not sure how much you know about this topic, so I will lay it all out for the sake of being complete.

Control groups are most often used in customer marketing promotions to “control out” the response of customers who “would have bought anyway”. This gives you a better picture of the true effectiveness of a promotion. PPC campaigns are generally thought of as customer acquisition tools, and so creating a “control” in the traditional sense is really not possible since you don’t have a pool of “non-customers” that you can measure against to see “who would have become a customer anyway”. Follow?

So really what you have with a PPC test of this nature is more like what would be called A / B / C testing; you’re not testing versus a real control, but testing against the “lack of an ad”, where A and B are PPC ads and C would be “the lack of A or B”. Then you would compare results.

Now the problem with this approach is (as far as I am aware) you cannot scientifically exclude an ad in a true random fashion and keep track of that exclusion set, so you really can’t measure the results of the ad versus “no ad”. Perhaps you could with some significant bid management / search engine API technology, but I’m not sure the engines would give up that kind of control - though I bet THEY have tested this at some level. It certainly would not be in their best interest to have people out there proving “no ad” was more profitable than “show ad”!

So here is what I did to approximate a true random sample exclusion test.

If you run campaign A on Monday, B on Tuesday, and C (no ad) on Wednesday, then start over with A on Thursday and continue this rotation, the following Monday B will run and on Tuesday C will run etc., so by the end of 3 weeks you will have A, B, and C data normalized by day - both campaigns and no campaign ran on every day of the week. Not a statistically pure method, but not horrible practice, for sure.

If the result spread (ad versus no ad) is significant enough, as it was in my PPC with Top 3 Organic link test, I’ll give up points in accuracy to get closer to the “directional truth”. Each of the top 30 search phrases where there was a top 3 organic ranking for the phrase was optimized in this way with the results very directionally consistent across all phrases. It was almost always more profitable to have a lower than #1 paid ranking when a top 3 organic ranking was present. Below the Top 30 phrases, some of the lower volume phrases produced inconsistent results which was probably a result of test method error / lack of frequency.

It probably would not be *technically* difficult to do a real random sample A / B / C test, if for example, Google allowed “No Ad” as an option under “Ad Variations” AND kept track of the visitors who searched with the PPC phrase who were *not shown* the ad. You would compare how many of these “no ad” visitors actually made it to your site / took the desired action versus those that were shown the ad, and then you would have a truly scientific test. But like I said, is it in the engine’s best interest to provide this functionality? Someday, it probably will be, as people get more sophisticated / are more accountable. But we’re probably a few years away from there…

Good luck with your blog !

K

Worse even than the blocking strategy is the “allocation strategy” where people simply “allocate” a certain amount of revenue to PPC and a certain amount to natural clicks based on some formula because “they work together”. This is by far the worst idea because it’s very easy to build a faulty case for PPC, especially if it is cannibalistic to natural. This is a very “old media” idea and symptomatic of the accountability problems in marketing these days.

I’m pretty much for any strategy, no matter what it is called, providing you can prove it is the most productive! Thanks for the comment Mark.  Let me know how the test comes out, if you need any help with thinking it through let me know; this is the way I did it.