Google’s Matt Cutts responds to the Bing “success rate” claims

It sounds a bit like the “Bing is copying Google” argument all over again, but Google’s Matt Cutts brings up some very interesting points in a small post via Google Buzz. In case you missed it earlier, we reported that Experian’s Hitwise metric was showing that Bing provided more accurate results than Google.

As Cutts points out, Hitwise relies on a metric called “search success rate”. The idea is that a successful search is completed once someone clicks on a search result and leaves the results page. This seems logical to us, but Cutts brings up some very strong arguments:

It sounds like Hitwise’s definition is “A successful search is defined as one where the consumer leaves the search engine after performing a search.” In another words, the user does a query and then goes somewhere else. That doesn’t sound the same as success to me; it just sounds like leaving the site.

Cutts then goes on to clarify his statement, asking whether the metric is able to tell the difference between someone successfully completing a search versus simply giving up on finding the information that they wanted. For instance, if someone performs a search on Bing and doesn’t immediately find what they want, then switches to Google, that is still considered a successful search since the user has left the results page.

Finally, there’s a measure of gaming the search metrics, as we’ve discussed before on TNW. Microsoft has a habit of sending traffic to Bing via its other portals (such as MSN.com). Click a link leading to a Kim Kardashian video on MSN, says Cutts, and you might find yourself at a Bing search result for “Kim Kardashian videos”. Cutts then asks the quite valid question of whether Experian’s metric would consider that to be a successful search as well.

There are more cases of how Bing has gamed the search system, including the similar case of photo slideshows. Want to view some photos? Each of those photos that you’re looking at is produced via a search query. By the logic applied to the previous scenario, each of these would be successful searches as well, even if you’re not especially planning on searching.

No matter how you look at it, there are valid points being raised by both sides. While Cutts’ point about a successful search does stand, it’s also worth noting that an unknown percentage of the results truly were successful searches. The difference, of course, being in how you define the term.

We’ll be following this mud war as it slings and let you know the results.