Mutually Exclusive Experiments

Definition

The Mutually Exclusive Experiment (M2E) feature allows you to run multiple experiments simultaneously on the same audience and with the same primary goal—without introducing UX conflicts or data bias.

➡️ To learn more about the benefits and potential pitfalls of this feature, see read our blog article: Mutually Exclusive Experiments: Preventing the Interaction Effect

Configuration

Creating an Exclusion group

To create a group:

1

Go to the Campaigns dashboard

2

Click the three dots next to the campaign.

3

Two options are possible: Add to exclusion group or Manage exclusion group, depending on whether the campaign is already part of a group.

4

To exclude a test campaign from another and create a group, select the option Add to exclusion group.

A modal appears. You are already located in the group creation option.

5

Give the group a clear name (for example, Homepage tests or Cart improvements) to help track included campaigns.

6

Open the dropdown menu to select the other campaigns to include.

The list is limited to 12 results by default, but you can use the search bar to find specific experiments.

You cannot add a campaign if:

  • It is already part of another group

  • It is live or has a start date set

  • It has already received traffic

To view only eligible tests, enable the Show available tests only option.

7

Select all the campaigns you want to include in the group.

8

Click Validate selection to add the selected campaigns to the group.

Traffic will be split across all campaigns in the group.

The more campaigns you include in a group, the more traffic is split—so reaching statistical significance may take longer. For example, two campaigns may double the time needed to gather reliable data.

9

To define traffic manually, toggle on Custom allocation. This is useful if some campaigns run on deeper pages with lower traffic.

Custom allocation limitations:

  • You cannot assign more than 100% total traffic

  • No campaign can receive 100% of the traffic

  • Allocation cannot be changed once the group is locked

10

Click Save to confirm the configuration.

Adding to an existing Exclusion group

If the exclusion group is not locked, you can add additional tests:

1

Go to the Campaigns dashboard

2

Click the three dots next to the campaign.

3

Two options are possible: Add to exclusion group or Manage exclusion group, depending on whether the campaign is already part of a group.

4

To exclude a test campaign from another and create a group, select the option Add to exclusion group.

A modal appears.

5

Select Add to existing group tab.

6

In the drop down list, select an available group (unlocked).

To view only eligible groups, enable the Show available tests only option.

7

Click Save

The test will be added to the group and its traffic allocation adjusted accordingly.

Group management

In the Campaigns dashboard, go to the Exclusion group tab to view and manage your groups. The number in parentheses indicates how many exclusion groups exist.

Click Manage next to a group to view or edit it, depending on its status.

Unlocked group

An exclusion group remains unlocked until at least one of its campaigns is live. While unlocked, you can:

  • Add or remove tests

  • Activate QA mode (by enabling IP address, URL or cookie targeting) without locking the group

Locked group

Once any campaign in the group is live, the group becomes locked.

By clicking on the manage button, you can still:

  • View its configuration

  • Delete the group if all tests are paused

You cannot:

  • Modify the configuration

  • Relaunch paused campaigns from a locked group that has been deleted (you must duplicate them first)

Deleting a group

To delete a group:

  • All campaigns in the group must be paused

  • If the group was unlocked, deleting it has no impact on campaign configuration

  • If the group was locked, paused campaigns cannot be relaunched to avoid allocation issue— the platform will invite you to duplicate the campaign instead.

Reporting indication

In the Reports tab, a message appears under the number of visitors to indicate the test was part of an exclusion group. Keep in mind that traffic volumes may differ from standalone experiments.

M2E allocation functioning

When multiple campaigns are mutually exclusive, the way traffic is allocated depends on their targeting and page coverage.

This diagram can be described in one sentence: if your targeted page isn’t the same for all campaigns of the group, part of the traffic landing on your website won’t be assigned to the desired campaign if visitors are bouncing.

On the Homepage, 33% of the traffic is allocated to campaign 1, and 66% to the 2 others. It doesn’t mean that 66% of the traffic will be assigned to the other campaigns, as among these visitors, some may not enter the targeting of the other campaigns.

Allocation scenarios

Targeting and page overlap
Result

Same targeting on the same page

Traffic is evenly distributed across all campaigns of the group. As your targeting conditions are the same, the exact same numbers of visitors will be assigned to each campaign.

Different targeting on the same page

Traffic is split evenly accross the group campaigns, but final traffic assignments vary by campaign targeting rules of the group.

Same targeting on different pages

As your Exclusion group contains several campaigns, when landing on one of the targeted page, your visitors, even if corresponding to the targeted audience, might be allocated to another campaign. Thus, if bouncing, they will never reach their allocated campaign and the number of visitors being assigned to the reporting will never be the same.

Different targeting on different pages

Use separate campaigns instead—M2E adds no value in this case

Use cases

#1 Avoiding data bias across similar campaigns

As a User I need to launch several tests at the same time on the same page. Those campaigns shouldn’t enter into UX conflict as they are acting on different elements but my primary goal is the same for all campaigns.

In order to avoid any data biases I’d prefer making them mutually exclusive and do a proper analysis at the end.

#2 Preventing UX conflicts

As a User I need to test different parts of my page and I’m afraid of UX conflict between those parts. Thus, to avoid any bad user experience I would better use the Mutually Exclusive Experiments feature and be more confident during the QA.

#3 Balancing traffic across different pages

As a User, I need to launch several tests at the same time on my website and all of them have the same primary goal. One campaign applies on the Home page and the other on the Product page. I know that I’ll have way more traffic on the HP than on the PP. Thus, I can use the Custom Allocation functionality of the Exclusion group to assign more traffic to my PP campaign than my HP campaign.

FAQ

What type of campaign can be made mutually exclusive?

You can add to the exclusion group experiments campaigns (test campaigns) such as A/B Test and Multipage Tests

How are cookies affected?

Once a visitor is assigned to a test in the group, the assignment is stored in a cookie. As long as the cookie persists, the visitor remains assigned to that test.

What happens when my visitor lands on my website?

The AB Tasty tag randomly assigns the visitor to one of the available tests in the exclusion group.

The visitor must meet the targeting criteria to see the test

What about QA?

We advise you to perform your QA before pushing a campaign to a group. If not, when using the QAA, you can assign yourself to the campaign by clicking on the "Force Display" CTA.

Also, when looking at ABTasty.results, you might not be assigned by your campaign if it's part of an exclusion group. For such reason, you can see a new rejected reason : exclusion_group_rejected. To be assigned to the campaign, you can either:

  • Remove your AB Tasty cookies and refresh the page (random allocation to campaign will be played again since you'll have a new visitorId)

  • Force the display to the campaign thanks to the QAA

Last updated

Was this helpful?