Q&A: Marti & Bruner discuss the problematic W3C proposed measurement standard ‘Attribution Level 1’
Don Marti, privacy expert, tech consultant, and W3C invited expert, and Central Control’s founder Rick Bruner, conducted this Q&A by over email. The introduction is Don’s, as are the questions, and the responses are Rick’s.
Introduction
The industry bodies promoting the attribution cartel under the banner of privacy are focused on solving the wrong problem. Even if the W3C Attribution proposal works exactly as intended, with strong privacy protections, secure implementations, and broad browser adoption, it will still produce reports that systematically mislead advertisers about what is actually driving incremental business outcomes. The result will be to reward the largest platforms, which control the browsers and would control the attribution reporting system, at the expense of other media channels, simply because they are easier to observe and measure within the system.
Even if fraud operators are unable to exploit the complexity of the Attribution proposal to add more costs, risk and noise (which fraud hackers have been able to do every other time complexity increased)...
Even if the Big Tech companies somehow decide not to unfairly favor their own products (for the first time ever)...
Even if the extra processing and resource consumption required for the Attribution proposal's mathematical goals turns out to be affordable and politically acceptable...
And even if the design is bug-free and browsers implement it perfectly...
Then reports that the attribution cartel can produce will still be actively misleading. The Attribution proposal is a complicated implementation of the same kind of statistical three-card monte trick that the Big Tech companies already use to over-report the effectiveness of their existing cross-context behavioral advertising (CCBA) services.
Left: Don Marti; Right: Rick Bruner
DON: Setting aside the specifics of the Attribution proposal for now, what are the fundamental limitations with the kind of attribution systems it aims to emulate?
RICK: Attribution systems fundamentally confuse observation with causation. They observe that an ad exposure occurred before a conversion and then assign credit within that sequence of events. But correlation does not imply causation.
The rooster crows before sunrise every morning. That does not mean the rooster caused the sun to rise.
The missing ingredient is the counterfactual: what would have happened if the ad had never been shown? Attribution systems cannot answer that question because they do not include valid control groups.
As a result, they systematically favor media that are easiest to observe and closest to conversion. They over-credit lower-funnel channels, reward targeting users already showing purchase intent, undercount upper-funnel media with delayed effects, and concentrate budgets into the largest browser-based platforms simply because those platforms generate the strongest measurable signals.
Privacy-preserving attribution systems do not solve these problems. They simply apply privacy constraints to the same fundamentally flawed measurement model.
One of the persistent problems with current attribution tracking systems is that they over-credit interactions that are closer in time to the sale. This is appealing to sellers of search, social, and app store ads, but a problem for advertisers trying to allocate budgets. Is there a way to apply some correction to existing attribution tracking, or are other kinds of studies needed in order to accurately weigh the impact of “lower funnel” and “upper funnel”?
There is no weighting scheme that converts attribution into causal measurement. First-touch, last-touch, multi-touch, and algorithmic attribution models are all still assigning credit within observed conversion paths without establishing a counterfactual.
Lower-funnel media naturally appear more effective because they intercept demand after it has already been created elsewhere. Search is the classic example. Brand advertising may create the demand, but the search click gets the attribution credit because it occurs closest to the sale.
The only reliable way to measure incremental impact is through randomized controlled experiments or strong quasi-experimental designs that compare exposed and unexposed populations.
The Attribution proposal would provide obfuscated measurement of ads on independent sites, while the Big Tech companies continue to collect individual measurement on their own properties. Is there a statistically valid way to compare the two different data sets?
No, not in a statistically clean way.
Independent sites would be measured through noisy, aggregated, privacy-constrained reporting, while the largest platforms would continue operating with deterministic user-level data inside their own properties. Those are fundamentally different measurement environments.
The result is asymmetric observability. Large platforms retain richer data and stronger measurable signals, while smaller publishers operate inside noisier systems with weaker signals that often disappear entirely under aggregation and privacy thresholds.
That naturally pushes budgets toward the biggest platforms, regardless of true incremental impact.
Machine learning systems tend to “learn” to target users already showing purchase intent, which results in attribution reports favorable to media channels that really created little or no incremental demand. Could the system be redesigned to avoid this effect, or are different kinds of experimental designs needed?
AI does not solve the correlation-is-not-causation problem. Machine learning is fundamentally a very fast and sophisticated form of pattern recognition on observational data. It can become extremely good at finding correlations, but that is not the same thing as measuring causal incrementality.
In fact, today’s AI systems are being trained on decades of flawed advertising measurement: attribution systems, weak quasi-experiments, surveys, and platform-reported ROI metrics that already over-credit lower-funnel media and in-market targeting. As I argued in my essay, “Enterprise AI is coming, and it’s about to learn all the wrong lessons about marketing effectiveness,” AI trained on flawed measurement systems will simply automate and amplify those same flawed assumptions.
This is not a bug in machine learning systems. It is what the systems are optimized to do. If attribution rewards conversions associated with observable ad exposures, AI systems will naturally learn to target users already likely to convert because that maximizes measured performance.
The solution is not more sophisticated attribution. It is experimental design. Randomized controlled trials remain the gold standard because they create valid counterfactuals. The encouraging part is that RCT-based measurement could itself be highly automated and incorporated directly into campaign execution systems at scale.
Attribution systems are a key part of the “dashboards” offered by Big Tech companies to smaller and less mathematically skilled advertisers. How can those advertisers without in-house data science expertise take the first step to understand the limitations of the analytics they’re being offered?
The first question advertisers should ask is: “Compared to what?”
If a dashboard claims advertising drove conversions, advertisers should ask what control group was used to determine whether those conversions would have happened anyway. Most attribution dashboards cannot answer that question because they are measuring observed associations, not causal lift.
This is not fundamentally a math problem. It is a measurement design problem.
Drexel University professor Elea McDonnell Feit tells her data science students there are “data takers” and “data makers.” Most analysts are given an existing data set and asked to find patterns inside it. But causality requires creating new data through randomized intervention and experimental design. You cannot calculate your way to causal certainty from observational data alone.
If advertisers and independent media companies want a real alternative to the attribution cartel, the industry should move toward standardized randomized controlled measurement frameworks that can be applied consistently across channels and platforms.
So hypothetically, let’s say one of the attribution cartel companies has an always-on mic on a smart TV, with enough speech-to-text capacity to recognize shopping and food ordering conversations. And an ML system “learns” to serve a pizza ad to households that are already talking about pizza. With the Attribution proposal as it stands now, is there any way to see when that’s happening, or is there just not enough data to tell that apart from an ad that actually drove a sale?
No. That scenario is really just an extreme version of what large platforms already do today through search behavior, browsing activity, location signals, and other behavioral data.
The Attribution proposal does not solve that problem because it still cannot distinguish between predicting a sale and causing a sale. If an ML system gets very good at finding people already likely to buy pizza, attribution systems will credit the ads for conversions that may have happened anyway.
The core problem remains the absence of a counterfactual. Attribution observes that an ad appeared before a purchase. It cannot determine whether the advertising actually changed behavior.
One approach you have suggested is randomized experiments using ZIP codes. If I have a rare pet or hobby, I might be the only person in my ZIP code who regularly reads some niche site. Would sites with smaller audiences benefit from some kind of ZIP code grouping?
Not yet, but they could.
Today, ZIP code targeting in digital media is often unreliable because it is inferred from IP addresses, mobile location signals, or multiple observed locations associated with a device. That makes it poorly suited for rigorous randomized experiments.
In my AdExchanger essay, “ZIP Codes: The Simple Fix For Advertising ROI Measurement,” I argued the industry should standardize around much more accurate “primary ZIP code” targeting tied to residential identity, while still preserving anonymity. The goal would be to make ZIP-based randomized controlled experiments practical and scalable across media channels.
Importantly, these experiments do not require identifying individuals inside the measurement process. Outcomes are measured in aggregate across randomized groups, not tied to user identities.
If sparsely populated ZIP codes create privacy concerns, that is a solvable methodological issue. Smaller ZIP codes could be clustered into larger randomized units before assignment into test and control groups, preserving both privacy and statistical validity.
More broadly, the need for sophisticated incrementality measurement depends on scale. Small advertisers can often infer incremental impact directly from obvious business changes after launching campaigns. The real challenge exists for medium and large advertisers operating across many channels, where the incremental contribution of any one channel may be subtle but still economically significant.
The big opportunity still not addressed by so-called “privacy-enhancing” advertising systems is that the mathematical “privacy” properties they offer are different from the “privacy” as experienced by people and described in the W3C Privacy Principles. Can you explain how your randomized experiment approach is both more accurate and more compatible with real privacy?
Most “privacy-enhancing” advertising systems still depend on observing and linking individual behavioral events across advertising and conversion activity. They may reduce how much data leaves the browser, but they preserve the underlying surveillance and attribution model.
Randomized geographic experiments take a different approach. Instead of tracking individuals, they compare outcomes across groups where advertising exposure was randomly varied.
That improves measurement because randomization creates a valid counterfactual. It also improves privacy because there is far less need for persistent identity, cross-context tracking, or detailed behavioral histories.
Observing Big Tech’s attribution claims is kind of like watching three-card monte games. We know that the average three-card monte player loses money. And in advertising, not only are small businesses in a crunch, we can see from Michael Farmer’s work that large advertisers are also growing more slowly than the economy as a whole. But when you look at a dashboard that claims to show ROAS, it’s like walking by a game and seeing the dealer keep paying out cash to players. What are we not seeing?
What you’re not seeing is the counterfactual.
A dashboard can show that people exposed to ads later converted. It cannot show whether those same people would have converted anyway. That is the core illusion behind attribution systems.
The problem is reinforced by incentives throughout the ecosystem. Media sellers are incentivized to maximize measured ROAS, not necessarily true incrementality. Agencies are rewarded for spending budgets efficiently in an operational sense, not for proving that every dollar created incremental demand. Internal marketing teams are often evaluated on growth, scale, and execution speed, not on conducting rigorous scientific audits of media effectiveness.
Meanwhile, attribution systems naturally reward channels that are easiest to observe and closest to conversion. That concentrates spending into the largest digital platforms, whose scale and observability produce the strongest measurable signals, whether or not they are generating the most incremental business value.
What disappoints me most is that industry bodies should know better by now. Thirty years into digital advertising, it has long been understood that attribution systems measure correlation, not causation. Yet instead of standardizing around randomized experimentation and true incrementality measurement, the industry continues institutionalizing attribution models that primarily benefit the largest platforms while systematically disadvantaging independent media and upper-funnel channels.
Ultimately, the advertisers themselves bear the cost. They may receive dashboards full of reassuring metrics, but if those metrics are based on flawed measurement, then budgets are being optimized around a statistical illusion rather than real business impact. Maybe it’s more like legalized gambling than three card monte, but regardless the house always wins.
How does a small business decision-maker get good enough at statistics not to fall for misleading attribution reports?
They do not need to become statisticians. This is not fundamentally a statistics problem. It is an eighth-grade science problem.
The key question is simple: how do you know the sales would not have happened anyway?
Attribution dashboards rarely answer that question because they generally do not include valid control groups or randomized comparisons. They observe correlations and assign credit within those correlations.
Business owners already understand this instinctively in other parts of life. If sales rise after running ads, that does not automatically prove the ads caused the increase. Other factors may have changed at the same time. The basic logic is not complicated.
In fact, smaller businesses often have the strongest incentive to get this right because owners and employees have real skin in the game. Some digitally native companies already think this way. They rigorously test landing pages, pricing, creative, and conversion funnels through experimentation. Extending that same mindset to media channels is a natural next step.
What is disappointing is that industry bodies should be leading this transition. They understand the methodological limitations of attribution systems and are in the best position to promote more rigorous, standardized incrementality measurement approaches that create a fairer playing field across the media ecosystem.