Gaming · Product
Tutorial A/B test
Reading a tutorial experiment on your warehouse, with Day-7 retention and first purchases as guardrails.
Simpler combat step vs. current tutorial
+12.1%
Significant winDay-1 retention
095% interval +6.3% to +17.9%
Simpler combat step
Current tutorial
control
Who it's for
Game designers and product teams who test the new-player experience in a feature-flag tool and need results they can trust.
What it solves
Analyze the tutorial test on the exposure data in your warehouse, with the same retention definition as your dashboards, and know when the result is safe to ship.
Questions it answers
Did the simpler combat step lift Day-1 retention?
Day-1 retention rose from 38.0% to 42.6%, a 12.1% lift (95% interval +6.3% to +17.9%).
Did Day-7 retention or first purchases move against us?
Day-7 retention and first-week purchases stayed within their guardrails.
- Is the lift significant, and how wide is the interval?
- Which platforms and acquisition channels responded best?
The impact
- Change the new-player experience on evidence, not on a hunch
- One retention definition for experiments and live-ops reporting
- Catch tests that lift retention and hurt monetization
The data it needs
- Exposure events from your experimentation or feature-flag tool
- Tutorial step and session events
- Purchase events
Read in place in your warehouse. Nothing is copied out.
Tutorial A/B test: frequently asked questions
Run this on your own data.
Start free on your warehouse, or book a demo and we'll show it on your data.