Measuring Ad Effectiveness

People like to complain about ads almost as much as they complain about the weather. Sometimes with good reason. However, advertisers need to know what they are getting for what they are spending.

Source unknown

In a simpler time, the advertiser could look at sales. However, with multiple sales channels (e.g., online, company store, and third party mail order or store), the sales figures they can see directly may not be complete. Further, resellers may be running their own ads or promotions that can distort sales figures and inflate the apparent results from the ads. Buyers may decide to make a purchase, but delay it to take advantage of a promotion. Remember the adage, “there’s always another sale.”

In the online world, advertisers can use a more direct assessment. They can track what people do — what you do — when exposed to an ad, to see if you are responding to it. The measures still aren’t perfect, but they’re better than a blind guess.

There are (at least) two schools of thought about how to assess online advertising effectiveness.

(1) Show the ads to everyone and use a statistical model to assess how much impact the ad is having. In this case, researchers capture user behavior online and use “balancing variables” such as demographics to eliminate the effects of factors that they are not trying to measure. In plain English, if women are more attracted to a brand then men, we would want to eliminate that as a factor in an overall assessment of an ad for the brand. We would want to “balance” the sample by gender to avoid distortion in the results.

(2) Alternatively, we can use a formal experimental design, in which Internet users are randomly assigned to a test group (exposed to the ad) or a control group (don’t see the ad).

Researchers at Northwestern University and Facebook have issued a report finding that the experimental approach is almost uniformly superior to the modeling approach. The modeling approach, used in the past by a number of major Internet companies, tends to overstate the impact of ads.

Part of the problem with the modeling approach is that it may not capture all of the factors needed to fully balance the sample. Some factors may simply not be measured, and there are limits to the number of factors that can be considered in a model.

If you manage the budget for advertising in a company, you need to know this. Your ads may not be working as well as you thought.

Sources: Institute for Operations Research and the Management Sciences. “How measurable is online advertising?.” ScienceDaily. ScienceDaily, 21 March 2019. <>

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.