Measuring Ad Effectiveness using Geo Experiments

Advertisers want to be able to measure the effectiveness of their advertising. Many methods have been used to address this need, but the most rigorous and trusted of these are randomized experiments, which involve randomly assigning experimental units to control and test conditions. Google have found that randomized geo experiments are a powerful approach to measuring the effectiveness of advertising.

Many advertising platforms allow advertising to be targeted by geographical region. In these experiments, we first assign geographic regions to test or control conditions and employ AdWords’ geo-targeted advertising capabilities to increase or decrease the regional advertising spend accordingly. The use of randomized assignments guards against potential hidden test/control biases that could impact the measurements. Our approach also accounts for seasonal changes that impact the volume and cost of advertising across the length of the experiment.

In this paper, they describe the application of geo experiments for measuring the impact of advertising on consumer behavior (e.g. clicks, conversions, downloads, etc.). This description includes the results of a geo experiment that Google research team ran for a Google advertiser.


Google uses geo experiments to help AdWords advertisers assess the effectiveness of their paid search advertising spend. These experiments measure the value of incremental ad spend, identify optimal bid levels, quantify the impact of online ad spend on offlines sales, etc. In these experiments, geos are randomly assigned to a control or treatment group and users in these geos are served, or not served, search ads accordingly. Typically, such one-off experiments require a custom setup, they are disruptive to the advertiser's marketing plan, and the results only reflect the competitive landscape at a snapshot in time. In this paper, we describe the adaptation of geo experiments to a framework of continuous measurement. This approach mitigates the issues stated above while providing more definitive results than equivalent one-off experiments. For example, results indicate that a continuous experiment that changes spend in 1/8th of the geos can achieve the same confidence interval as an analogous one-off (non-continuous) experiment that changes ad spend in 1/2 of the geos.