Dynamic traffic allocation is based on a statistical research topic called “multi-armed bandit”. The goal of this algorithm is to limit the loss due to the lowest-performing variations.
However, when the campaign has just been launched, the performance of the variations is not known yet. When data collection begins, the performance of variations is known progressively, but these measures are not fully significant.
At this point, it would be an error to send all traffic to the apparent winner. However, some formulas can help to compute the right amount of traffic to send to each variation with the following goal: sending more traffic to the variation that could win, but not too much, otherwise, there would not be enough traffic to ensure the measurement of the supposed losers.
The solution is to periodically change the allocation regarding previous data, following a statistical formula that ensures the goal mentioned above.
Dynamic allocation is a technique that can be applied when a classic A/B Test is not possible because of lack of time, or traffic. This technique is only applicable in a given context (typically with stable KPI). If you have enough traffic, and are not sure that the KPI to optimize is stable then you should prefer the classic A/B test.
Before switching to dynamic allocation, you need to define at least a primary goal in the Goal step of the test creation flow. The Dynamic allocation algorithm is exclusively based on this primary goal. This means that it detects the best-performing variation based on the increase in conversions of the primary goal. For further information, please refer to the .
The primary goal can be any type of goal except Browsing metrics (Revisit rate, Pages per session, and Bounce rate). To activate dynamic allocation, click the Change/switch to Dynamic allocation button at the top of the page.
A confirmation message is displayed:
By confirming, the traffic allocated to each variation will be determined based on the predictive algorithm and on the primary goal, you configured for your test. At this point, you will not be able to manually adjust the percentage of traffic allocated to each variation. However, you are still able to define the percentage of traffic you want to allocate to your test and the percentage of traffic you want to leave untracked.
If you did not select a primary goal, you will not be able to switch/change to dynamic allocation. In this case, you will be asked to define at least one primary goal and you will be redirected toward the goal step.
Once the test has been launched, you can access the reporting anytime, as you would for any campaign with static allocation.
From the test dashboard, tests using dynamic allocation are easily identifiable thanks to the icon.
Once your test has been launched with dynamic allocation, you will not be able to perform the following actions:
Change the primary goal
Add, remove or duplicate a variation within the test
Switch back to static allocation
If you want to perform any of these actions, you need to pause the test and duplicate it.
Dynamic Allocation can be very useful in the following cases:
When you want to optimize micro conversions that are expected to occur within a short period after the user has been exposed to a variation. For instance, e-commerce, prefers the Add to cart goal rather than the transaction, as the primary goal.
When you have very little time to run a test (ex: testing the headline of a media article, testing the message of a countdown banner).
When you have really low traffic on the page you want to test, but still want to do some optimizations. Dynamic allocation carries out optimization even before reaching enough data to be fully significant. This would not be very interesting because the traffic is low. But it may become interesting if carried out on a high number of pages. For instance, if you have low traffic on each product page, you cannot carry out a classic A/B test. But it may be interesting to implement multiple dynamic allocation tests, one for each product page. The gain will be low for each page, but it can be more significant when multiplied by the number of pages.
We recommend not using dynamic allocation in case you want to optimize a website/application where the visitors will be different over time. Since allocation changes over time, the primary goal should be stable during the experiment. Example: a meal-ordering website, or a taxi company, customers and behaviors will not be the same at noon and 3am
When you have a lot of variations to test (more than six), dynamic allocation enables you to quickly identify the lowest-performing variations to run the test on the most relevant ones.
When creating a test or a personalization campaign, you need to define the percentage of visitors assigned to the variation(s) (in case of a test) or scenario(s) (in case of a personalization) and to the original version of your website.
To learn about the traffic allocation step of the campaign creation flow, refer to Campaign flow: Traffic Allocation step.\
For each individual visitor, we compute what we call a seed, a unique alphanumeric string that will be persistent in time. This seed, also unique for each campaign, will be used to generate a score between 1 and 100 and then the same logic as before applies.
As the seed is computed each time, we do not need to store the result anymore.
This allocation system doesn't work under the HTTP protocol as we are using cryptographic tools that are only available under a secure protocol.
If the tag detects it is on an insecure environment, it will automatically fallback to the old mechanism.
Let's take the example of a 50/50 campaign. If a visitor gets a score of 51, they will be exposed to the variant (a score above 49).
When Even Allocation is active, the platform restricts you from manually setting a variation's traffic allocation to a value exceeding the original allocation.
To know more about this behaviour, read our .
If you decide to change the allocation from 10/10 to 50/50, the allocation will be computed again and their score will now place them in the original. The visitor will switch from one variant to another.
This table shows the different categories of users, representing your entire website traffic. For each category, information on what the users will see is displayed:
It is highly not recommended to change the allocation while a campaign is running. Doing so might severely impact the reliability of your results, even more now with the new system. We strongly advise you to duplicate your campaign and start a fresh new one unless you consider the risk is low (the campaign is still in QA, only a few visitors have seen it until now, etc ..).
For example, let’s say you’ve implemented a ‘risky’ test and want to limit risk-taking. In this case, you can limit the number of visitors tracked and so that 40% of visitors aren’t tracked. That way, you reduce the test sample and limit the visibility of your new version(s). This means that these 40% of untracked visitors will see the original version of your website but won’t receive a cookie in their browser related to the test. Their actions on the website won’t be tracked unless you decide to increase the percentage of tracked visitors. The percentage of visitors that you track depends on the volumetry of your website. For example, if your website has over 10M unique visitors per month, you can choose to track only 20% of the entire traffic (and limit the test sample to 20%). However, if you have around 200k unique visitors per month and decide to track 20% of the traffic, you won’t reach reliability quickly enough to determine whether you can increase the percentage of tracked visitors. For more information, refer to .
\
Even Allocation is a feature designed to promote sound experimentation practices and ensure the reliability of your A/B, Multipage, and Multivariate tests.
This article explains how Even Allocation works and how it can help you achieve more accurate test results.
The Even Allocation feature prevents you from allocating a higher percentage of traffic to a variation than the original traffic allocation in your campaigns. When enabled, this feature automatically adjusts traffic distribution to adhere to even allocation principles. This helps avoid skewed data and ensures that each variation receives a fair share of traffic, leading to more trustworthy results.
🚦 Type of traffic
🍪 Visitor’s cookie campaign information (ABTasty cookie)
💬 Comment
👻 Untracked visitors
campaignId=[campaignId] value=-1
Visitors who see the original version of your website but whose actions will not be taken into account.
🎯 Tracked visitors (% of unique visitors of your campaign)
campaignId=[campaignId]
value/variationId=0 or variationID of the variation
The original version of your website or a new variation/scenario of your website
🏛 Traffic allocated to original
campaignId=[campaignId] value=0
Visitors within the test sample who see the original version of your website.
🆕 Traffic allocated to variations
campaignId=[campaignId]
variationId=[variationID of the variation]
Visitors within the test sample who see a new version of your website (or one of the new versions if there are several variations/scenarios).
Traffic Restriction: When Even Allocation is active, the platform restricts you from manually setting a variation's traffic allocation to a value exceeding the original allocation.
Informational Message: When the feature is enabled, an informational message appears on the allocation page, reminding you that variation traffic cannot exceed the original traffic.
Campaign Duplication: If you duplicate a campaign with uneven allocation while Even Allocation is enabled, the duplicated campaign's traffic will be reset to an even distribution. A warning message will alert you to this change during the duplication process.
Even Allocation promotes statistically sound experimentation by preventing common allocation mistakes. Here's why it's beneficial:\
Prevents Skewed Results: Uneven allocation can lead to biased results, making it difficult to determine the true impact of each variation.
Encourages Best Practices: By enforcing even allocation, the feature encourages you to follow established CRO (Conversion Rate Optimization) best practices
Ensures Fair Traffic Distribution: Even allocation ensures that each variation receives a representative sample of your website traffic, improving the validity of your test
Here are a few examples of how Even Allocation impacts traffic allocation:\
Scenario 1: You have an A/B test with two variations. With Even Allocation enabled, the maximum allocation for each variation is 50%.
Scenario 2: You have a Multivariate test with three variations. Even Allocation ensures that each variation receives approximately 33% of the traffic.
Scenario 3: You have a Multipage test with three variations. Even Allocation suggests that each variation receives approximately 33% of the traffic, but you ultimately decide to allocate 33% of the traffic for the original, 33% for Variation 1, and 28% for Variation 2.
Scenario 4: You attempt to allocate 70% of the traffic to the original and 15% to the variation in an A/B test. With Even Allocation enabled, the platform will prevent you from doing so. The allocation will stay even between the original and the variation. So you can have up to 50% traffic allocated to both the original and the variation. Or choose to have 15% traffic allocated to variation. The system will automatically allocate the same percentage (15%) to the original, and the remaining traffic (70%) will be untracked.
If you have any questions about Even Allocation or need help managing your account settings, please contact your Customer Success Manager.
But you can allocate more traffic to your variation(s). Drag the Traffic allocated to the variation slider towards the right or add more than 50% of traffic into it.
You can set the percentage of your traffic that will be allocated to each version.
This means that 40% of untracked visitors will see the original version of your website but will not receive a cookie in their browser related to the test. Their actions on the website will not be tracked unless you decide to increase the percentage of tracked visitors.
The percentage of visitors you track depends on the volumetry of your website. For example, if your website has over 10M unique visitors per month, you can choose to track only 20% of the entire traffic (and limit the test sample to 20%). However, if you have around 200k unique visitors per month and decide to track 20% of the traffic, you will not reach reliability quickly enough to determine whether or not you can increase the percentage of tracked visitors.
For more information about the calculation of the traffic, you would test regarding your needs, refer to our Sample Size Calculator article in the AB Tasty blog.
To get more information about reliability and the chance to win in the reports, please visit our specific guide Campaign reports.
We recommend keeping the same traffic allocation throughout the campaign duration.
Heads up ⚡
If you change the traffic allocation while your campaign is live, only the new visitors will be affected by this change.
Visitors who have already been affected by a variation (before this change) will keep on seeing the same variation.
If you want to change the traffic allocation of the test to assign 100% to the variation, you think is the highest-performing after analyzing your report, you need to create a patch campaign from the current test. Otherwise, people who have a cookie from another variation or the original version will continue to see the original version of your website.
You must use the duplication feature to transform your winning variation into a fresh new campaign patch.
If you allocate 100% of your traffic to your personalization campaign, you will not be able to make a comparison with the original version. We recommend leaving at least 10% of your traffic on the original version (depending on the amount of traffic on your website) to be able to control the performance of your experience, make a comparison with the original.
Set up the traffic allocation on the AB test from step 5 Traffic allocation of the creation flow.
By default, 100% of your visitors will go through the campaign and the traffic will be split equally between the original and the number of variations of your test. In our example with one variation, 50% of the traffic will be allocated to the original and the other 50% to the variation.
You can always change this configuration by limiting the traffic allocated to variations at the campaign level (Traffic allocated to variations slider in the bloc "Global traffic") or allocate more traffic to a specific variation (variations sliders in the bloc "Variations traffic").
By default the traffic allocation for AA tests is locked to 100%
Set up the traffic allocation on the Multipages test from step 5 Traffic allocation of the creation flow.
By default, 100% of your visitors will go through the campaign and the traffic will be split equally between the original and the number of variations of your test. In our example with one variation, 50% of the traffic will be allocated to the original and the other 50% to the variation.
This is exactly the same principles and flexibility than for AB tests.
You can always change this configuration by limiting the traffic allocated to variations at the campaign level (Traffic allocated to variations slider in the bloc "Global traffic") or allocate more traffic to a specific variation (variations sliders in the bloc "Variations traffic").
Set up the traffic allocation on the Multivariate test from step 5 Traffic allocation of the creation flow.
A multivariate test is made up of several “sub-tests” and can therefore be used to test a range of combinations of variations. You therefore need to configure the traffic allocation for each sub-test.
By default, 100% of your visitors will go through the campaign and the traffic will be split equally between the original and the number of variations of your subtests. In our example with one variation in the first sub-test, 50% of the traffic will be allocated to the original and the other 50% to the variation.
This is exactly the same principles and flexibility than for AB tests, but at a sub-test level.
You can always change this configuration by limiting the traffic allocated to variations for each sub-test (Traffic allocated to variations slider in the bloc "Global traffic") or allocate more traffic to a specific variation (variations sliders in the bloc "Variations traffic").
Once you've set-up your traffic allocation for the first sub-test, don't forget to do the same for the other ones by using the dropdown.
The best practice remains to keep a perfect iso repartition for each subtest.
The Traffic allocation step is not available for Patch and Multipages patch campaigns, meaning that 100% of your traffic will see your patch.
Set up the traffic allocation on a Simple personalization from step 5 Traffic Allocation of the creation flow.
By default, 100% of your global traffic is allocated to your experience. Although, we recommend leaving at least 10% of your traffic on the original version (depending on the amount of traffic on your website) to be able to control the performance of your experience and make a comparison with the original.
Set up the traffic allocation on a Multipage personalization from step 5 Traffic Allocation of the creation flow.
Each percentage of the targeted traffic is assigned to a page. Do not forget to set up traffic allocation for each page of your campaign using the dropdown.
By default, 100% of your global traffic is allocated to your experience but you can change this configuration.
Set up the traffic allocation on a Multiexperience personalization from step 5 Traffic Allocation of the creation flow.
Each percentage of the targeted traffic is allocated to an experience. Do not forget to set up traffic allocation for each experience of your campaign using the dropdown.
By default,100% of your global traffic is allocated to your experience but you can change this configuration.
If you prefer to use the Dynamic allocation option, you don't have to set-up your traffic allocation. For more information please refer to the specific article.
In case you need support, follow the instructions given in the articles below:
Head’s up
To prevent misshapen allocations our Even allocation feature prevents you from setting a variation’s traffic allocation to a value higher than the original.
Head’s up
To prevent misshapen allocations our Even allocation feature prevents you from setting a variation’s traffic allocation to a value higher than the original.
Head’s up
To prevent misshapen allocations our Even allocation feature prevents you from setting a variation’s traffic allocation to a value higher than the original.