Why You Can’t Trust Comscore’s Search Market Share Numbers

Nearly every media source in the United States uses Comscore’s data to report on Google, Bing, Yahoo!, AOL and other engines. Even the US government, when analyzing Google’s potential monopoly status used these figures. Yet, this data is completely misleading. Here’s what Comscore says Google’s current market share looks like:

Yet, many folks who run analytics on their websites see data more like this:

Granted, this is an extreme example (seomoz.org’s search traffic, whose audience is very Google-biased), but the principle’s similar. I haven’t seen a site in the last 3 years whose search engine share contained >10% from Bing or Yahoo!, and often seen numbers below 5% for the two combined.

What’s going on? Does Google have 65% market share, 97% market share or something in between? I’ll try to explain.

When Comscore reports that Google has 65% of the searches they’ve observed in their monitoring network, they’re not directly lying, nor is their sample necessarily biased (though I suspect it may be, at least a little). Instead, it’s wrapped up in what and how they count.

One key item to take into account is how Comscore describes the search engines in their reports:

When they say “Microsoft Sites” and “Yahoo! Sites” they don’t just mean the search engines Bing and Yahoo run. They mean any search that leads to search results on any site owned by Microsoft or Yahoo!. That means if a user performs a search like this:

Or like this:

Or like this:

Yahoo! Shopping results, MSN Autos, news searches from either all count in Comscore’s evaluation of total searches of those company’s networks. Broadly, this means that a lot of searches that happen internally on Microsoft and Yahoo! properties by users who never leave those networks are being counted.

There’s also a great deal of search data missing from Comscore’s analysis due to mobile searches. On the qSearch methodology page, there’s not a single mention of mobile device tracking, nor is there any indication that Comscore’s currently collecting data from mobile queries (which may account for 10%+ of all searches in the US). Here’s SearchEngineLand’s Greg Sterling on Comscore’s market share data:

 These figures don’t reflect mobile search queries or market share. In mobile Google enjoys a roughly 95 percent share in the US market.

When you combine network bias with the lack of mobile search data, the numbers are a bit less surprising. I suspect there’s even more behind the curtains here, including possible sampling bias, Comscore missing large chunks of data from certain types of Google queries, or something else.

Nevertheless, if you want more accurate search market share data, there are, thankfully, a few good sources.

The first is Statcounter, which has referral data for 3 million+ websites, and a nifty interface that shows the delta between market share data over time:

Statcounter’s 80% market share for Google, 9.5% for Bing and 8% for Yahoo! strike me as far more credible than Comscore’s. They also match up nicely to Chitika, an advertising network with referral data from ~250K websites. They don’t have an interactive tool, but they do release regular updates on search engine market share:

Those numbers are somewhere between Comscore’s and Statcounter’s, but they do note clearly that it includes desktop search only, not mobile devices (where Google’s market share is far more dominating). I suspect Chitika’s bias to sites that serve advertising has an impact on the Google search numbers as well.

My final note on this issue comes in the form of an anecdote. Around 2007, back when Google’s actual market share was much closer to 60%, I was working with a media site as an SEO consultant. This particular site noted more search volume from being in top rankings on Yahoo! for terms like “wwe wrestling news” and “nascar results” than they received from similar positions in Google. Weird though it may seem, Yahoo! may have actually been getting more queries for those terms than Google, thanks to their audience profile.

The takeaway is simple – depending on your niche and the keywords you target, you may find dramatically more/less market share is controlled by a given engine. The results certainly wouldn’t be as dramatic today, but users of Yahoo! and Bing still have different demographic and psychographic profiles than Google.

Bottom Line: Don’t take Comscore’s data at face value. And don’t panic if you’re getting 90% of your search traffic from Google – it doesn’t necessarily mean you’ve done something wrong with Bing or Yahoo!.