By Marketers love to talk about A/B testing. Heck, I’ve said it too. It sounds prudent. Responsible. “Test before you commit! Don’t waste resources!” But the longer I work in marketing, the more I believe most A/B tests are a waste of time — at least for small and medium-sized businesses. More often than not, these tests are just procrastination disguised as optimization. What marketers really need is conviction.
Unless you’re running a site with hundreds of thousands of users or you’re in the ballpark of hundreds of millions of dollars in revenue, the data you’re getting probably isn’t even statistically significant. Who cares if 845 people liked your blue CTA while “only” 692 clicked the purple one? That’s not a meaningful insight; that’s noise. Because if we’re defending best practices, we ought to be consistent. And this means you need to look for at least 1,000 conversions per variant in an A/B test in order to yield a reliable result.
At some point, you just need to make a decision. Better yet, make the right decision based on your understanding of your audience. You know, ahem, through audience research.
Setting up an A/B test isn’t free. Even a “simple” test takes time: someone has to scope it, build the variant, QA it, add tracking, and monitor the results. Then you have to let it run long enough to reach statistical significance. That takes days (if you get solid traffic), but more likely, weeks. And for what? A slightly better click-through rate on a button?
Too many teams burn cycles on tests that won’t move the needle. The opportunity cost is real — that’s time you could have spent launching the next high-impact campaign. Or at least writing a blog post that pisses off a bunch of people. (Like this one.)
Marketing isn’t just data and dashboards. It’s taste, judgment, context. You don’t need a test to decide whether your hero image should feature a person or a product. You need a clear point of view, a unique value proposition, and a deep understanding of who you’re talking to.
This is the core of what we do at SparkToro: through our software, we help marketers know their audience so well that they can make bold decisions without second-guessing. If something flops? No big deal. Change it back. Try something else. But don’t waste two weeks “testing” something your gut already knows.
At SparkToro, as a marketing team, we don’t run A/B tests on our homepage copy. We talk to our audience, listen to what they say, and write what we believe will resonate. And if it doesn’t? We fix it fast. That level of agility beats a weeks-long test every time. However, we do A/B test our email subject lines. But that’s because we’re reaching 70,000+ people, and it’s dead simple. All we have to do is set up a quick automation and wait an extra two hours for our email service provider to move forward with the winning variant. (Thank you, MailChimp!)
Here’s an uncomfortable truth: a lot of A/B testing is just fear in disguise. We’re afraid to make the call. Afraid to be wrong. So we spin up a test instead. It feels like progress — but really, it’s avoidance. It’s the marketing version of tweaking your Notion dashboard instead of doing the actual work.
Shipping imperfect things will teach you more than endless testing ever could.
Fine, sometimes A/B testing makes sense.
Of course, there are exceptions. If you’re running ad campaigns with millions of impressions, managing an ecommerce site with high purchase volume, or you’re operating at a scale where a small variant actually does make a measurable impact on revenue, testing different creatives or layouts can lead to meaningful gains. Especially when the sales cycle is shorter, making immediate, data-driven changes can make a real impact.
But if you’re not a high-volume, high-spend business? You don’t need to act like you are.
Make the change. Move on.
You don’t need to test everything. You just need to know your audience, trust your instincts, keep it moving, and most importantly, be willing to change course. The best marketers aren’t the ones who run the most tests. They’re the ones who move quickly with conviction and the humility to be wrong.
So next time you feel the urge to test two button colors… maybe just pick one. Then get back to doing work that actually matters.