Sequential Testing Alerts

Configuring Sequential Testing Alerts

Sequential Testing Alerts (STA) helps you detect underperforming variations early by continuously monitoring your experiment’s primary goal against a chosen sensitivity threshold.

When the algorithm identifies a likely losing variation, STA can:

• Send an email alert, so you can react quickly, or

• Automatically pause the campaign, to limit negative impact

This is especially useful for server-side A/B Test campaigns, where a variation can affect critical backend logic (pricing, eligibility, routing, recommendation algorithms, etc.)

Configuration

To enable Sequential Testing Alerts:

1. Open your A/B Test campaign

2. Go to Step 6 Advanced options

3. In Sequential Testing Alerts, choose:

   1. The automation type

   2. The sensitivity threshold

1

Choose the automation type

You can select what happens when a variation is detected as underperforming:

• Receive an email alert (no automatic interruption) Use this option when you want human validation before acting.

• Automatically pause the campaign and notify by email Use this option for higher-risk campaigns, where limiting downside is critical.

2

Choose the sensitivity threshold

The sensitivity threshold defines how quickly STA reacts to negative signals.

• High - safest option More alerts, faster detection of losing variations.

• Balanced - default middle ground Reasonable detection time with fewer false positives.

• Low - advanced option Fewer alerts, but losing variations can take longer to be detected.

Prerequisites and limitations

• STA can only be enabled if your campaign has a Primary goal configured.

• STA decisions are based on the primary goal performance and the selected sensitivity level.

Learn more

• Detection levels: Sequential Testing Alert article

• FAQ: Sequential Testing FAQ

If you don’t select a Primary goal, Sequential Testing Alerts can’t be activated.