Why Retail Media Networks Grading Their Own Performance Is the Real Measurement Problem

Retail media has become one of the fastest-growing categories of ad spend, and the way its performance gets reported has a structural problem that's easy to overlook because none of it looks dishonest on the surface: the entity reporting how well a campaign performed is also the entity that sold the ad space, and there is no independent party in between checking the math. Amazon reports its own definition of sales attributed to an ad. Walmart Connect reports gross merchandise value calculated its own way. A grocery retail media network splits its own in-store and online attribution using its own logic. Each of these numbers is a genuine measurement of something, and each of them comes from a platform with a direct commercial incentive to report a number that justifies more ad spend on that platform specifically.

Self-Reported Metrics Aren't Lying, They're Just Not Neutral

None of this requires bad faith to produce a systematic bias. A closed measurement environment, using last-click logic and a definition of "attributed sale" that the platform itself controls, tends toward metrics that look favorable almost by construction, not because anyone is fabricating numbers, but because the platform designs, reports, and audits its own scorecard with no outside check on the definitions being used. A brand comparing Amazon's reported ROAS against Walmart Connect's reported ROAS isn't actually comparing two measurements of the same thing, it's comparing two different platforms' self-graded assessments of their own effectiveness, using methodologies neither platform is incentivized to make directly comparable to a competitor's.

The Cross-Retailer Problem Is Worse Than the Single-Platform Problem

Even accepting each platform's own number at face value for a moment, a brand running campaigns across Amazon, Walmart, Instacart, Target, and a grocery network simultaneously faces a harder problem underneath the individual metrics: there's no shared identity graph connecting a customer's exposure to an ad on one platform with a purchase that happens on a different one. A shopper who sees a Pinterest or retail-media ad on one platform and completes the purchase through a different retailer entirely is invisible to both platforms' attribution systems, each of which can only see the slice of the journey that happened on its own property. This isn't a minor edge case at the scale retail media spend has reached, it's the default condition of running a multi-retailer strategy, and it means the sum of what every platform individually reports as attributed sales routinely overstates what actually happened in aggregate, since overlapping credit and invisible cross-platform paths both push the combined number upward.

A Concrete Illustration of Overlapping Credit

Consider a shopper who sees a sponsored product ad on Amazon, doesn't click, then searches directly for the brand on Google two days later and buys through the brand's own site instead. Amazon's system may still count this as an ad-influenced view-through outcome in its own reporting, crediting the sponsored placement for at least partial impact. Separately, if that same brand also runs Google Shopping ads and the purchase happened to touch a Google ad somewhere in that path, Google's own attribution system might independently claim credit for the same conversion. Add a third platform, a retargeting ad the shopper saw on a completely different site in between, and there can be three separate systems each reporting they contributed to one single sale, none aware of the other two, each grading a shared outcome as if it were entirely their own. Sum every platform's self-reported contribution across a real multi-channel campaign and the total routinely exceeds total actual revenue, sometimes by a wide margin, simply because overlapping credit isn't something any individual platform's dashboard is built to detect or correct for.

Why Incrementality Is the Question Everyone Struggles With

Industry surveys consistently find that a large majority of retail media advertisers cite incrementality, whether the ad actually caused a sale that wouldn't have happened anyway, as their single biggest measurement challenge, while only a small minority feel genuinely effective at measuring it. That gap exists precisely because incrementality requires a controlled comparison, a holdout group that didn't see the ad, compared against one that did, and a closed platform reporting its own attributed sales has no structural reason to build and surface that comparison honestly against its own revenue interest. The platform's own dashboard answers "how much activity happened near this ad," which is a fundamentally different, and much easier, question than "how much of this activity wouldn't have happened without the ad."

Why This Compounds as Retail Media Budgets Grow

Retail media ad spend is growing at a pace well ahead of most other digital channels, which means the absolute dollar value riding on unverifiable self-reported numbers grows every year even before accounting for how many additional retail media networks a brand adds to its mix. A brand running retail media on one platform faces one instance of this measurement bias. A brand running it across five, chasing the fastest-growing channel across every major retailer, faces five separate self-graded scorecards, none directly comparable to the others, informing a single combined budget allocation decision. The scale of the problem isn't static, it's compounding at exactly the rate the channel itself is being adopted, which makes this a worse problem to leave unaddressed with each passing budget cycle rather than a fixed cost that gets easier to absorb as the channel matures.

What to Do Without a Shared Identity Graph

Waiting for retail media networks to build interoperable, independently-audited measurement isn't a realistic near-term plan, since none of them have a commercial incentive to make their own performance easier to compare unfavorably against a competitor. The more realistic path for a brand spending meaningfully across multiple retail media networks:

• Run platform-specific incrementality tests directly, rather than trusting the platform's own reported figure. A geo or audience holdout test run independently of the platform's dashboard, even a simple one, produces a genuinely comparable answer that a self-reported ROAS number can't.
• Treat each platform's self-reported metric as a directional signal within that platform, not a cross-platform currency. Comparing Amazon's reported number against Walmart's reported number as if they measure the same thing produces a false sense of precision neither platform's methodology actually supports.
• Anchor final budget decisions to total business outcomes, not the sum of platform-reported attribution. Overall revenue and share of a category, tracked independently of any single platform's dashboard, is the check against a combined attributed-sales number that's structurally likely to overstate total impact once overlapping and invisible cross-platform paths are accounted for.
• Push for standardized measurement contractually where spend justifies it. At meaningful budget levels, some retail media networks will negotiate access to independent measurement partners or more transparent methodology documentation, but this generally has to be requested rather than offered by default.

This Is the Same Pattern as Every Other Walled Garden

None of this is unique to retail media specifically, it's the same structural dynamic every closed advertising platform has always had, applied to a newer, faster-growing category of spend. Search and social platforms have run into the same self-reported measurement skepticism for years, which is exactly why the broader shift toward independent incrementality testing and multi-layer measurement stacks, rather than trusting any single platform's own dashboard, has been building across digital advertising generally. Retail media isn't introducing a new problem so much as reintroducing a familiar one, at a moment when the category's growth rate makes the cost of not addressing it larger with each passing quarter.

The Uncomfortable Bottom Line

A retail media network's dashboard isn't wrong to trust because it's dishonest, it's wrong to trust as a complete picture because it was never built to be a neutral referee of its own performance against a competing platform's ad inventory, and no amount of dashboard sophistication changes who's grading the test. Treating each platform's reported number as one input, checked against independent incrementality testing and total business outcomes, is the only version of this that scales honestly as retail media spend keeps growing faster than the measurement infrastructure built to independently verify it.

None of this means retail media isn't worth the investment, the channel's genuine growth reflects real performance for a lot of brands, not just favorable self-reporting. It means the number on any single platform's dashboard should carry roughly the same weight in a budget conversation as a vendor's own case study: a useful, self-interested data point, not the final word on whether the spend is actually working.