Most ecommerce brands do this wrong!

Key Takeaways

you know what drives me crazy when I see Brands run a test for a few days get excited about early results and completely change their email strategy based on bad data then they wonder why their revenue drops a month later let me show you exactly how we validate our test at Lenco it's simpler than you think but you have to get three things right first and this is absolutely critical you need enough data let me show you why look at these two variations variation a had 711 recipients and variation B had 58 can we really trust that comparison actually no the difference in the sample size is too big here's what you want to see at least a thousand people in each variation and roughly the same number in both groups in both variations why because that's when percentages actually mean something the second thing is time your test needs to run for at least 2 weeks ideally a month look at what happens when you don't do that week one variation a was crushing it with a 15% conversion rate the client wanted to stop the test right there thank God we didn't by week four variation B had completely taken over winning with a 23% conversion rate why because different types of customers shop at different times some people buy right after payday others wait for their credit card cycle to reset