Creating Experiments
Learn how to create and manage A/B tests and feature flags with TokPulse.
Overview
Experiments in TokPulse allow you to:
- Run A/B tests on your store
- Implement feature flags
- Test different configurations
- Measure impact on key metrics
Experiment Types
A/B Tests
Traditional A/B tests compare two or more variants to determine which performs better.
Example: Testing different button colors
- • Control: Blue button
- • Treatment: Green button
- • Metric: Click-through rate
Feature Flags
Feature flags allow you to enable/disable features for different user segments.
Example: New checkout flow
- • Control: Current checkout
- • Treatment: New checkout flow
- • Rollout: Gradual (10% → 50% → 100%)
Creating an Experiment
1. Basic Setup
curl -X POST http://localhost:3001/api/experiments \
-H "Content-Type: application/json" \
-H "X-Org-ID: your-org-id" \
-d '{
"key": "checkout_flow_test",
"name": "Checkout Flow Test",
"description": "Test new checkout flow vs current",
"allocation": 100,
"variants": [
{
"key": "control",
"name": "Current Flow",
"weight": 50,
"configJson": "{\"flow\": \"current\"}"
},
{
"key": "treatment",
"name": "New Flow",
"weight": 50,
"configJson": "{\"flow\": \"new\", \"steps\": 3}"
}
]
}'Best Practices
Experiment Design
- Clear Hypothesis: Define what you're testing and why
- Single Variable: Test one change at a time
- Statistical Significance: Ensure adequate sample size
- Duration: Run for at least one business cycle
Configuration
- Fallbacks: Always provide fallback behavior
- Validation: Validate configuration JSON
- Testing: Test variants before launching
- Monitoring: Set up alerts and guardrails