Can I A/B test Hyvä against Luma before full cutover?
Yes, and for stores above $10M GMV I usually recommend it. Two patterns:
- Sub-domain split test. Hyvä on
hyva.yourstore.com, Luma onwww.yourstore.com, send 10-25% of traffic to the Hyvä sub-domain via a paid traffic source (Meta / Google Ads) for 2-4 weeks. Measure conversion, AOV, bounce, INP, error rate. If Hyvä outperforms (it usually does, +8-25%), do the full cutover. Cost: ~$2k, $5k of paid traffic for a clean signal. - Geo split. Cut over one country / region first (typically the smallest by GMV), watch 4-6 weeks, then cut the others. Lower-risk pattern for multi-region brands. No extra licence cost.
Things to know:
- A/B testing Luma vs Hyvä on the same URL via Optimizely / VWO doesn’t work cleanly, the two themes have completely different DOM + CSS so the A/B tool flickers visibly.
- You need analytics consistent across both: same GA4 / Matomo setup, same conversion definitions, same UTM handling. I set this up as part of step 4 (Build).
- The split-test “Hyvä loses” outcome is rare but real, usually means a specific Magento extension didn’t port cleanly and broke a critical flow. Diagnose, fix, re-test.