How to create a Multivariate Test

A Multivariate test enables you to test combinations of changes simultaneously on your website. You can change several elements on a page simultaneously (e.g., the color and wording of a CTA) and identify which of the possible combinations performs best according to the goal you wanted to reach (e.g., increasing the number of clicks, the number of pages viewed, decreasing the bounce rate). You can then apply these changes directly to your website. Unlike an A/B test, which involves testing each hypothesis in a different test, a multivariate test allows you to run hypotheses at the same time and to find the best combination of all. The purpose of multivariate tests is to measure the interactive effects between several supposedly independent elements (e.g. page title and visual illustration). A subtest contains variations that will be tested independently across multiple combinations. For example, if your goal is to test a simple CTA, you might want to create three subtests: Color, Shape, Wording, with as many variations of colors, shapes and wordings as necessary. AB Tasty will create all the possible combinations and display them randomly to your traffic.

To learn more about the differences between each type of test, refer to the Types of campaigns article. If you don’t know which type of campaign to choose, our virtual assistant Ally can help you.

To create a Multivariate test, go to Web Experimentation, click the Create button and select Multivariate Test.

👉 To learn about the different ways to create a campaign, please refer to How to create a campaign.

The number of combinations created is calculated as follows: [Number of variations of the 1st subtest]*[Number of subtests]. The more subtests with variations you configure, the more traffic you need to allocate the different combinations. In this case, the readiness of your campaign and data reliability takes more time to be reached than a classic A/B test.

Configuration

In Multivariate Tests, you can’t add goals with our standard goals setup step - it is not available for this type of test. Only the transaction goal is available and preselected. This is also why, in the editor, you can't add action trackings or tracking widgets.

Step 1: Main information

The Main Information page is the first step toward your new Multivariate Test campaign.

We recommend establishing a hypothesis of your test according to the following model: If I apply [this change on my webpage] to [this audience], then [it will impact] and enables to enhance [this goal]. And to add it in the hypothesis field.

By default, only one subtest is created, but you can add as many subtests as necessary. For each subtest, enter the URL you want to load in the editor. In most cases, you must enter the same URL for all subtests.

👉 For more information on how to configure this step, refer to How to set-up Main Information step.

Step 2: Editor

You have two ways to create content for your campaign in AB Tasty:

With the visual editor

For each subtest, you can add as many variations as you want. For example, if your subtest one is related to the color of a button. In a variation 1, you can test the blue color and in a variation 2, a red color.

👉 To learn how to use the Visual Editor, discover options, and dive into our widget libraries, please read the Visual Editor Guide

With the code editor

The Code Editor enables you to declare your code (JavaScript, CSS) without having to load the Visual Editor. The Code Editor is the best solution for developers who want to save time. To learn how to use the Code Editor, please read the Code Editor Guide.

👉 To learn how to use the Code Editor, please read the Code Editor Guide.

Step 3: Targeting

Be careful when entering the targeted pages (in the Where section) of your test to ensure that it will display correctly on the pages you want to target (e.g. Product pages). To trigger the test only for visitors using a specific device, use the Device criterion in the triggers section. When the Who, How and Where sections have the same configuration for each sub-test, you can use the Replicate targeting option.

👉 To learn how to configure targeting, please read How to setup a campaign targeting.

Step 4: Traffic allocation

All users must be tracked. Each percentage of the targeted traffic is assigned to a subtest.

👉 To learn how to configure Traffic allocation, please read Traffic allocation.

Step 5: Advanced options

The advanced options are not mandatory.

• Enable the third party tool(s) you have connected to AB Tasty to send your test data to your tool.

• Select a loading method for your campaign.

👉 To learn how to configure Advanced options, please read Campaign flow: Advanced Options.

Step 6: QA

The QA of the campaign is one of the most important steps before launching your campaign into production. The QA allows you to verify and test in real condition :

- the targeting configuration - the tracking - the modifications you've made

👉 To learn how to use the QA Assistant, please read QA Mode & QA Assistant.

If you want to launch your A/B Test at a specific hour on a specific day, and/or if you want to pause it at a specific hour on a specific day, see our guide on scheduling.

Use cases

Multivariate tests can be used in the following cases:

🖊️ Action / modification	🎯 Goal(s)
Testing the wording and color of a CTA	Increasing the conversions on a CTA
Testing the layout of several elements on a page (coupon code field, CTA, etc.)	Optimizing the shopping cart