A/A Testing vs A/B Testing: What's the Difference?

An A/B test shows two different versions of a page or element to separate groups of visitors to find out which one performs better on a defined metric, such as conversion rate.

An A/A test shows two identical versions of the same page to separate groups of visitors - there's no real variation between them, so there's no winner to find. Its purpose isn't to compare content at all; it's to check whether the testing tool's traffic split, randomization, and result tracking are actually working correctly.

Because both versions in an A/A test are identical, any meaningful difference the tool reports between them signals a problem with the testing setup itself - broken randomization, a tracking error, or a sample size too small to trust - rather than a real effect. Running an A/A test before a program of A/B tests establishes that the tool can be trusted and gives a baseline read on natural variance between two randomly split, otherwise-identical groups, which A/B test results can then be judged against. Skipping this step doesn't just risk missing a setup problem once; a broken testing environment quietly biases every A/B test run on it afterward, which is why A/A testing is treated as a one-time validation step rather than something worth running on an ongoing basis once trust in the tool is established.