A/B Testing
Question: Does a challenger channel outperform the control in conversion and revenue efficiency? Data: customer-level acquisition outcomes by channel, including cost, conversion, and revenue. Method: two-proportion z-test for conversion lift plus confidence intervals and supporting revenue distribution diagnostics.
Dataset
800 Records
Customer acquisition data
1. Analysis Name
A/B Testing
This page compares a control channel against a challenger, testing whether conversion lift is statistically credible and whether revenue supports the same decision.
2. Problem Context
What problem this page answers
The business problem is to decide whether a challenger should replace or complement the control. The page separates observed performance, statistical significance, and commercial impact so the recommendation is auditable.
3. Observed Data
Observed cohort outcomes and descriptive summary
The first chart shows the actual converted and non-converted counts for each cohort. The table summarizes sample size, conversion, revenue, and cost before any inference is applied.
Variant Summary
4. Workflow
How the experiment answer is built
The workflow checks observed lift, quantifies statistical uncertainty, and then reviews revenue behavior to confirm whether the apparent winner holds up commercially.
Build cohorts
Keep control and challenger distinct and compare like-for-like channel groups.
Test conversion lift
Apply a two-proportion z-test at the selected confidence level.
Validate business impact
Compare revenue distributions to avoid picking a statistically better but commercially weaker option.
Experiment Verdict
-
Winning Variant
-
Test Statistics
5. Conclusion
Recommended experiment answer
Why this is the best answer