Everything We Know About Ad Rotation & Testing in AdWords for 2018

Ever since ETAs were released, Google has recommended having three or more ads per ad group. Over the course of the last six months or so, however, a new debate has emerged surrounding the topic of ad rotation and testing. This has spawned countless articles on industry blogs and news sites and even a compelling presentation at HeroConf London (“Debunking Ad Testing” by Martin Röttgerding). Much of the information is conflicting, and it’s hard to know when to abandon the tried and true methods for the cutting edge ones that could revolutionize your results.

At Rocket Clicks, we have learned to approach new features with what I would call “skeptical optimism,” believing that new features are rarely “one size fits all.” There are notable exceptions – like ad extensions – where the benefits clearly outweigh the risks, but for everything else, we want to understand those risks and the potential performance lift involved in order to develop a structured way of testing new features that will maximize their benefit for our clients.

We have spent months learning about the new ad rotation settings and having discussions with our Google team about the mechanics involved.

Initially this process was very frustrating, with only a handful of poorly developed case studies and anecdotal stories in support of adopting 3+ ads per ad group and using the “optimize” setting. In most of the official material we reviewed, scientific examples were lacking, and most results were shown for cases where advertisers went from one ad per ad group to three or more ads per ad group.

That is very different from how we do things, which led us to feel that the new recommendations by Google were aimed at the same audience that would use AdWords Express, or perhaps large agencies managing several hundred or thousand accounts at the same time with little individual attention and a lot of automation. We were confused about what the best course of action was heading into 2018, but we knew that it was mission-critical to get ads right.

We kept digging and eventually struck paydirt when we had the opportunity to speak with a Product Specialist at Google. Finally, we confirmed some of our hypotheses, determined the best way to test this change in ad testing moving forward, and even figured out when NOT to use 3+ ads and optimize.

This is the information that we feel has been missing all along, and because we felt so lost and frustrated for months, we want to share the plan for doing ad testing and ad rotation we’ve made with everyone else who may have also felt like we did this year.

First and foremost, when shouldn’t you use 3+ ads per ad group? This may not be a good fit if:

• You need to guarantee a specific ad text will show, like during a promotion
• Your campaign is limited by budget
• Your campaign is very low traffic*
• You have campaigns split out by device (and/or match type)

*We learned that for low traffic campaigns, the algorithm running ad rotation effectively lowers the statistical significance threshold to allow for faster changes due to the sample size required to have a high degree of confidence. This behavior approximates testing decisions many advertisers would make anyway under similar circumstances, but it’s important to note that you can’t be scientifically confident in what the results are telling you below 95% significance.

So what if your campaign has cleared all of the above checks and you want to try the new approach, but aren’t ready to blanket the settings in all campaigns?

This was our hang up with many accounts. We were too risk averse and feared performance volatility.

However, the purported benefits of the new ad testing methodology, included CTR and conversion lift which sounded too enticing to pass up!

We decided to try and understand what the mechanism of action could be – in other words, without access to any real 3rd party data, how could this increase in CTR and conversions be happening through either more ads per ad group or the “optimize” ad rotation setting?

What we hypothesized was that either the ad relevance component of quality score was raising on average with the addition of the new ads, or that the algorithm behind the “optimize” ad rotation was doing something to either increase impression share or better match ads to users.

We discussed many potential scenarios and outcomes including using many keywords in one ad group or only broad match types leading to diverse query matching in an ad group. We wondered if a sophisticated advertiser who already had a quality score of 10 would also see the uptick in performance by following the recommendation, or if we would observe it going from 2 ads to 3 instead of 1 to 3. There were more questions than answers!

After all of this debating and whiteboard drawing, we ultimately received confirmation that the primary mechanism of action is in fact ad relevance, one of the components of quality score. However, there is another outrageously awesome benefit of the “optimize” ad rotation setting which made us believe at last that we needed to try it.

According to the Google Product Specialist, not only does the algorithm running “optimize” match ads to users using the entire breadth of 1st party data available to Google, but it also has a feature that can INCREASE the number of auctions you are even entered in!

Think about that for a minute: more auctions actually increases your overall “pie” of impressions available, so not only can you potentially get more impression share, you can also get more impressions, too!

The Plan

If you are now likewise convinced, you should use Drafts & Experiments in order to split test moving forward.

You will need enough traffic to get good data quickly, so choose a high volume campaign. Google says it will take 2-4 weeks to have reliable data, but the experiment performance dashboard will let you know when the results are in. From there, your base campaign will remain in tact, with your original “rotate” ad rotation setting and two ads per ad group.

You can choose to divert 50% of traffic to your experiment campaign, and there have the “optimize” ad rotation setting and three or more ads per ad group. In order to isolate variables, the only difference in ad copy should be the addition of the new ad(s) in the draft campaigns.

Google does recommend that once you adopt the new “optimize” ad rotation setting, you will want to include more diverse ads in each ad group, testing on a more thematic basis in order to have ads that could match to diverse audiences. This might be a bit of a change if you have been A/B testing very methodically for a long time like we have.

So, assuming your campaign is out of experiment mode, you will want to assess an ad’s performance in comparison to its ad group averages and your CTR and conversion goals. We learned that if a significant amount of time passes, and the ad is receiving few impressions, it is a pretty good signal that you can safely pause or remove that ad.

That’s great for getting rid of ads, but what if you want to introduce and test new themes in a structured way? This is where we would recommend using the ad variations tool found within the Drafts & Experiments tab, which has received a facelift in the new interface. This tool allows you to test specific components of your text ads in a structured way, and will give you statistical data that can inform future decisions for new ad themes you are exploring.

My hope is that you are now slightly less confused about ad rotation and testing, understand the potential benefits, and have a clear plan for how you can test it and roll it out in your accounts moving forward.

If you’ve found this helpful or have any comments or questions, please respond in the comments below or ask me on Twitter (@Benjamin_Page or @RocketClicks.