The reporting may show differences between the percentage of visitors assigned to each variation. For example, you have defined a 50/50 traffic modulation, and your report shows a different split.
These differences may be caused by one of the following reasons:
When a visitor lands on the website for the first time, they are randomly assigned a variation (or the original version). They will remain assigned to the same variation if they visit the website again.
However, if there is not enough traffic on the website, its distribution between the variations will not be even. For instance, out of 10 visitors, seven could have been assigned to variation 1 and three to variation 2. With 100 visitors, you could have 60 who see variation 1 and 40 who see variation 2. The more visitors your page has, the more the campaign will tend towards a 50/50 distribution.
You may see a difference of up to 12.4% from 1,000 visitors; 4.32% from 10,000 visitors; and 1.1% from 100,000 visitors. The difference is only up to 0.37% once you reach a million visitors.
You can check if traffic allocation was changed via the campaign history tab in your campaign reporting:
If the allocation was changed after the campaign’s launch, it is more likely that traffic modulation won’t be respected because the visitors who had already been assigned to a variation (or to the original version of your page) won’t be impacted by this change.
These visitors already have the information stored in the AB Tasty cookie of their browser, so they will always see the same version of your page that they were initially shown. Traffic modulation changes will only apply to new visitors.
For example, if your campaign was originally 50/50, and you change it to 70/30, there is a chance your campaign will never actually reach 70/30.
For more information, refer to this page: I deactivated a variation in the traffic allocation step, but still see it online.
Last updated
Was this helpful?
Was this helpful?
