A/B Test Setup

使用场景

• Designing experiments with proper statistical rigor
• Calculating required sample sizes for desired power
• Setting up feature flag configurations for gradual rollouts
• Defining primary and guardrail metrics for experiments
• Analyzing experiment results and making ship/no-ship decisions

Design an A/B test for the following change:

Feature: [describe the change]
Hypothesis: [what you expect to happen]
Primary metric: [what you're measuring]
Current baseline: [current metric value]
Minimum detectable effect: [smallest meaningful improvement]
Traffic: [daily active users or events]

Provide:
1. Experiment design (control vs variants)
2. Sample size calculation with 80% power, 95% confidence
3. Expected duration to reach significance
4. Guardrail metrics to monitor
5. Feature flag implementation code snippet
6. Analysis plan (when to check, how to decide)

推荐模型

兼容工具

模态

相关 Skills