This page is a curated library of essays, research papers, case studies, and industry analysis on advertising experiments and measurement. It highlights work from academics, practitioners, and platforms that advances evidence-based marketing and rigorous incrementality testing.
HBR: “It's time to close the experimentation gap in advertising: Confronting myths surrounding ad testing”
HBR examines why marketers continue to underuse experiments despite their proven value, drawing on interviews and surveys across multiple industries. The authors identify common myths that hold teams back and outline strategies for building a culture that relies on experimentation. Colin Campbell, Julian Runge, Kenneth Bates, Stacey Haefele, Neeraj Jayaraman. Link (paywall)
Northwestern Kellogg: “Is Your Digital-Advertising Campaign Working? If you are not running a randomized controlled experiment, you probably don’t know”
A great business article summarizing research Kellogg did with Facebook (see below in Academic sources) demonstrating why “quasi-experiment” methods, with “synthetic controls,” along with other types of sophisticated “observational” statistical inference techniques were incomparably worse than randomized controlled trials (RCT) in reliably estimating “lift” from advertising campaigns. Link
Northwestern Kellogg: “Close Enough? A Large-Scale Exploration of Non-Experimental Approaches to Advertising Measurement”
Highly influential paper by Brett R. Gordon, Robert Moakler, Florian Zettelmeyer comparing randomized controlled results of 663 campaigns to quasi-experimental models for the same, finding that qausi-experimental results do not “reliably estimate an ad campaign's causal effect.” Link
HBR: “Marketers Underuse Ad Experiments. That’s a Big Mistake.”
Julian Runge, a behavioral scientist, writes in HBR how few companies run true advertising RCTs—only 12.6% in a study of 6,700+ firms—despite clear performance gains for those that do. The article explores why experimentation is underused, from organizational inertia to reliance on legacy models, and outlines the cultural and structural changes needed to make ad testing a core part of marketing strategy. Link
ANA Genius Award (Short Video): “Sam’s Club: Excellence in Marketing Analytics”
The Association of National Advertisers (ANA) honors Sam’s Club with a Genius Award, recognizing the excellence in their programs of randomized controlled trials and uplift modeling for maximizing the incremental return on marketing dollars. The three-minute video is a great case study on incrementality measurement best practices. Tony Rogers, Chief Member Officer at Sam's Club, states clearly in the video the value of those programs: "It's allowed us to have a lot more certainty and a lot more precision around ROAS (return on ad spend) and renewal rates of our members." Link
MediaPost: “Why The Most Important New Industry Acronym May Be RCT”
By Joe Mandese, editor in chief, reporting on the announcement of Central Control’s project with the Advertising Research Foundation and other partners, “RCT21,” writes: “The method, known as “randomized control testing” — or what will likely become the acronym du jour on Madison Avenue for the next several years, RCT — goes beyond classic marketing and/or media-mix modeling and so-called attribution systems, to scientifically correlate and measure the effect of actual advertising.” Link
The Correspondent: “The new dot com bubble is here: it’s called online advertising”
A harsh, widely read article, shining a light on inherent flaws in how most advertisers measure the effect of their digital advertising investments.
Google Thnk: “Measuring Effectiveness - Three Grand Challenges”
Excellent business paper, 49 pages, the first “grand challenge” of which is “Incrementality: Proving cause and effect.”
AdWeek: “The Current State of Digital Measurement by Attribution Is Flawed”
By Nathan Woodman, founder of Proof, formerly Chief Data Officer of Havas, laying out how inherent biases in popular observational methods of analytics do not serve advertisers’ interests well.
Broadcast & Cable (NextTV): “A Scientific Approach to TV Ad Measurement”
Gaurav Shirole, VP of Ad Measurement at Roku, shares some great advice on how advertisers can be more disciplined in running randomized experiments for TV and digital video, including this: “Start with the scientific method: develop a specific hypothesis to test. Ad measurement works best when marketers are solving a specific business problem.” Link