31st August 2009
Google Results Prefetching in Firefox/Mozilla
It appears that, some time ago, Google removed details of results prefetching from its Webmaster guidelines while continuing to implement results prefetching in its search results.
If you haven't a clue what I'm talking about, the Wayback Machine has the original Google Webmaster help on this topic, which I'll paste here verbatim in order to make it searchable (Wayback Machine pages aren't indexed by search engines):
Results Prefetching Questions
1. What is "results prefetching," and how does it impact my site?
On some searches, Google uses a special tag supported by Firefox and Mozilla to instruct the browser to download the top search result before the user clicks on the result. When the user clicks on the top result, the destination page will load faster than before. This tag is only inserted when it is likely that the user will click on the first link.
For example, when a Firefox user searches for [stanford], Google includes the following tag in the results HTML:
The official Mozilla Link Prefetching FAQ describes the behavior of this tag in detail.
Prefetching may impact your site because the prefetch request will happen whether or not the user clicks on the result, so it may result in additional traffic to your web server. Google only inserts this tag when there is a high likelihood that the user will click on the top result, but clearly this heuristic is not right 100% of the time.
2. Can I distinguish prefetch requests from normal requests?
Yes, as described in the Mozilla Link Prefetching FAQ, prefetch requests include the additional HTTP header
3. I want to block/ignore prefetch requests. What should I do?
To block or ignore prefetch requests (from Google and other web sites), you should configure your web server to return a 404 HTTP response code for requests that contain the "X-moz: prefetch" header.
What else do you need to know about results prefetching?
Google only issues the prefetch code when they are very confident that searchers will click on the #1 result (as in their example, a search for stanford). Most times, particularly for more "normal" sites (i.e. not Stanford), Google won't issue the code. So you may never see this on your own site.
However, it's worth being aware of this issue because if you do see a prefetch in your raw logs you'll want to know why; and because, depending on how you calculate conversions, the fact that a page is prefetched but never viewed by a searcher may significantly affect your conversion tracking and monetisation on that page. I'm surprised that Google removed this info from their Webmaster help.