Reporting Insights with RevenueIQ

RevenueIQ is an advanced feature that makes it easier to interpret the results of transactional A/B tests. By introducing a new set of revenue-focused metrics, RevenueIQ gives you a clear, statistically sound view of the revenue impact of each test variation. This empowers you to make faster, more confident, and more profitable decisions based on real revenue projections, rather than relying solely on traditional metrics like conversion rate or average order value.

Access RevenueIQ

1

Log in to AB Tasty

Go to your AB Tasty dashboard and log in with your credentials.

2

Navigate to Your Campaign Reports

1. From the main menu, select the campaign (A/B test) you want to analyze.

2. Open the report for a campaign that uses a transaction goal (i.e., a goal that tracks purchases or revenue).

3

Open the RevenueIQ Insights

1. In the campaign report, look for the Transaction metrics section.

2. Click on the Insights tab to access our smart analysis and recommendations.

3. The left panel provides a summary and recommendations based on RevenueIQ’s statistical analysis.

4. The right panel displays visualizations of key revenue stats and projections.

The Insight tab will be displayed by default for all accounts that haven’t selected another tab as the default.

Statistical reports details

Below are the report details

Revenue stats tab

1) Projected Uplift per Visitor

This metric shows the most probable incremental revenue that a winning variation could generate per visitor. It helps you understand the direct financial value of each user and assess your potential gain or loss if the variation is launched.

How it's calculated: The median value of the Revenue Uplift Confidence Interval, divided by the number of unique visitors in the variation.

We add our secret sauce to: "consolidate" and get rid of "extreme" values to give more accurate results !

2) Projected Uplift per month

This metric shows the most probable incremental revenue that a winning variation could generate per month. It is a revenue projection per month based on median value of revenue uplift confidence interval. It helps you understand the direct financial impact of each variation per month if you ship it live on your page traffic.

How it's calculated: The median value of the Revenue Uplift Confidence Interval, divided by the number of days the campaign ran and multiplied by 30 and projected on all traffic

3) Revenue Uplift Confidence Interval

This metric provides a range of potential revenue gains, offering a clear view of the most likely, pessimistic, and optimistic scenarios. This allows you to quantify your risk and potential reward more precisely.

4) Revenue Uplift Chance to Win

This metric provides the percentage probability that a specific variation will generate a positive revenue uplift compared to the original. A high percentage (e.g., >95%) indicates a high probability of a profitable outcome.

AOV stats tab

1) AOV Confidence Interval

This metric provides a range of potential AOV gains (in currency), offering a clear view of the most likely, pessimistic, and optimistic scenarios. This allows you to quantify your risk and potential reward more precisely.

2) AOV Chance to Win

This existing metric has been fully integrated into the Bayesian engine, improving its accuracy.

Important notes

• Bayesian Only:

  • revenueIQ stats are powered by our new Bayesian statistical engine.

  • They are not available in frequentist mode.

• Baseline Traffic is Key: If a report shows a "0" for a metric in a variation even if transactions occurred, it is because there was insufficient traffic in the baseline (the original version). It is essential to ensure a consistent flow of traffic to your baseline to get accurate and actionable results.

• RevenueIQ will be available for all tests that started after 1st of august

• RevenueIQ will not be available if traffic allocation has been changed by the campaign lifecycle

Learn more

• Analysing transaction goals