Mutually Exclusive Experiments
Last updated
Was this helpful?
Last updated
Was this helpful?
Mutually Exclusive Experiment (M2E) is a feature enabling you to launch several tests at the same time, on the same targeting, with the same primary goal without being afraid of any UX conflict or data biases.
To learn more about benefits and traps of M2E, you can read
Everything starts in the test dashboard.
When clicking on the 3 dots of a campaign, you’ll see 2 options. Either “Add to Exclusion group” or “Manage Exclusion group” depending if your test is already part of a group or not.
To exclude a test campaign from another, select the option “Add to exclusion group”. When done, a modal appears.
In that modal, you’ll be able to select between the creation of a new group starting with this test OR to add this test to an already existing group.
You can name your group to better keep track of the test you’re excluding (ie: “Home page”, “Add to cart improvement”, etc.).
Once done, you can select the different tests you want to make mutually exclusive.
Opening the dropdown will give the list of all tests of your account (limiting it to 12 at the first render). You can also search for specific tests should you know the name.
It might be possible that you can’t select a test in the dropdown. There are several reasons for that:
The test is already part of a group
The test is already live
Tests can only be part of one group at a time and a test which has already received some traffic can’t be part of an exclusion group.
Adding a test to an exclusion group decreases its global traffic allocation, meaning that if you would be able to change the allocation in the middle of a test, data would be biased.
To see only the test you can add to the current exclusion group, you can click on the “Show available tests only” option.
Then, clicking on Done will add that test to the exclusion group and split the global allocation per campaign.
You can now Save that group and continue your activities!
The more tests you’ll add to a group the more your traffic will be split between campaigns! Meaning that if you were waiting 14 days to have significant results before, you might have to wait 28 days now with 2 campaigns in a group. Please, use that feature wisely and ask your CSM if you have any doubt.
In case you wanted to apply a specific allocation per campaign, a “Custom allocation” option is available.
Choosing it enables you to customize the allocation for each campaign. It can be useful in case you have campaigns applied on deeper pages.
Be aware of those restrictions when using Custom allocation:
You can’t put more than 100% traffic
You can’t put 100% traffic on a specific campaign
Custom allocation can’t be changed once a group is locked
If your Exclusion group already exists and isn’t locked, you can add your test to it by clicking on the option “Add to existing group”.
You’ll be able to select available groups (groups which aren’t locked yet).
You can still see available groups only by ticking the box “Show available groups only”.
Selecting a group and Saving will directly add your test to the group and you’ll be able to play with its configuration.
In the test dashboard, you can see a tab “Exclusion group” where you’ll have an overview and be able to manage all of them.
The number between parentheses indicates the number of exclusion groups created.
By clicking on the manage button, you’ll be able to view the group configuration or play with it depending on the group status.
An unlocked group is an exclusion group in which tests have never been pushed live. As soon as a test in a group is pushed live, the group is locked and its configuration can’t be edited anymore.
So clicking on the Manage button on an unlocked group will enable you to add or remove tests from the group.
Activating the QA of a test (meaning that you toggle ON the IP Address and/or cookie parameter), your test becomes IN QA and you can still add it to a not locked exclusion group. Also, moving a campaign to IN QA once it's in a group doesn't lock the group.
A locked group is a group in which a test has been pushed live. As soon as the group has been locked, its configuration can’t be changed.
Clicking on the manage button will enable you to view the current group configuration.
You’ll still be able to delete the group even if all tests are paused.
You can delete a group at any time as soon as all the tests inside are paused.
If the group was not locked, there would be no impact on campaign configuration.
If the group was locked, you’ll not be able to relaunch campaigns which were already launched, to avoid any allocation issue.
In case you want to do so, the platform will invite you to duplicate the campaign.
In the reporting, you’ll see a notification under the number of visitors on the Activity Headers to remind you that your test was part of an exclusion group and the global traffic of your test isn’t the same as usual.
This diagram can be described in one sentence: if your targeted page isn’t the same for all campaigns of the group, part of the traffic landing on your website won’t be assigned to the desired campaign if visitors are bouncing. On the Homepage, 33% of the traffic is allocated to campaign 1, and 66% to the 2 others. It doesn’t mean that 66% of the traffic will be assigned to the other campaigns, as among these visitors, some may not enter the targeting of the other campaigns.
Below some examples of conditions:
Targeting conditions are the same and on the same page
When landing on your targeted page, all your traffic will be split between the different campaigns of the group. As your targeting conditions are the same, the exact same numbers of visitors will be assigned to each campaign.
Targeting conditions aren’t the same but on the same page
When landing on your targeted page, all your traffic will be split between the different campaigns of the group. But since the targeting conditions are different, campaigns won’t have the same number of visitors assigned to them.
Targeting conditions are the same but not on the same page
As your Exclusion group contains several campaigns, when landing on one of the targeted page, your visitors, even if corresponding to the targeted audience, might be allocated to another campaign. Thus, if bouncing, they will never reach their allocated campaign and the number of visitors being assigned to the reporting will never be the same.
Targeting conditions aren’t the same and not on the same page
In that case, running your campaign through an exclusion group doesn’t make any sense. You can launch them “separately”.
As a User I need to launch several tests at the same time on the same page. Those campaigns shouldn’t enter into UX conflict as they are acting on different elements but my primary goal is the same for all campaigns.
In order to avoid any data biases I’d prefer making them mutually exclusive and do a proper analysis at the end.
As a User I need to test different parts of my page and I’m afraid of UX conflict between those parts. Thus, to avoid any bad user experience I would better use the Mutually Exclusive Experiments feature and be more confident during the QA.
As a User, I need to launch several tests at the same time on my website and all of them have the same primary goal. One campaign applies on the Home page and the other on the Product page. I know that I’ll have way more traffic on the HP than on the PP. Thus, I can use the Custom Allocation functionality of the Exclusion group to assign more traffic to my PP campaign than my HP campaign.
You can add to the exclusion group experiments campaigns (test campaigns) such as A/B Test and Multipage Tests.
When the user lands on your website, a random test among your exclusion group is assigned to him. As soon as his cookies aren’t erased, he will still be assigned to the same test.
When your visitor lands on your website, the tag randomly assigns him to one of the tests of the exclusion group. Note that the visitor must belong to the targeting rules to see the test.
We advise you to perform your QA before pushing a campaign to a group.