Matt is head of Google's Webspam team. We talked together quite a lot in 2001 when Matt was working on the Google Webmaster Guidelines. Matt had seen my talk on spam and cloaking at the 2001 Search Engine Strategies Conference in San Francisco, and had read my White Paper on The Classification of Search Engine Spam. He was particularly keen on my ideas of "Doing it for humans". These ideas eventually found their way into Google's Quality Guidelines as
Make pages for users, not for search engines. Don't deceive your users or present different content to search engines than you display to users, which is commonly referred to as "cloaking."
Since 2001, Matt and I have communicated regularly and have always seen eye-to-eye on issues of spam. That is, until now. I can't help but feel that Matt's stance on paid links and the application of the rel=nofollow attribute is the start of a slippery slope for Google.
Back in 2001, when we agreed on "Doing it for humans", the principle behind this was that as a Web publisher you shouldn't need to do anything specifically for search engines in order for them to crawl, index or rank your content. If you found yourself doing something specifically for search engines, that was the time to ask yourself whether you were spamming.
Fast forward to 2005 and Google introduced the rel=nofollow attribute; a contributor to that post was Matt Cutts himself. The best places to use this tag, according to the post, are
the actual links that other people can create [...] for instance, only the links within comments and the link immediately after "Posted by:" would get the rel="nofollow" attribute.
This seemed like a great idea at the time. With hindsight, it marked the introduction of a tag that was specifically designed to affect search engine ranking algorithms:
At the heart of our [Google's] technology is PageRank™, a system for ranking web pages ... [that] relies on the uniquely democratic nature of the web by using its vast link structure as an indicator of an individual page's value.
By asking publishers to label links with rel=nofollow, Google was giving those publishers the power to affect PageRank. Nothing more and nothing less.
Search engines offer tags and other techniques to prevent crawling or indexing of Web pages - notably robots.txt files and the robots meta tag. But not since the ill-conceived and now much deprecated meta keywords tag has a search engine offered the ability to control rankings, in the way that the rel=nofollow attribute does.
So, the problem with rel=nofollow is that it's "Doing it for search engines", not the "Doing it for humans" that Matt Cutts and I agreed on back in 2001. Matt covered this in his post:
That same philosophy would mean that you wouldn’t create a robots.txt file (users don’t check those), never make any meta tags (users don’t see meta tags), never create an XML sitemap file (users wouldn’t know about them), and wouldn’t create web pages that validate (users wouldn’t notice). Yet these are all great practices to do.
However, there are two big differences between rel=nofollow and these examples quoted by Matt:
- rel=nofollow is not designed to affect crawling, or indexing, which are naturally of interest to Webmasters (as it's their site and their content); but ranking, which is the preserve of the search engine (as it's their algorithm).
- Failure to know about or deploy robots.txt, meta tags or sitemaps is not search engine spam. However, Matt is saying that failing to use rel=nofollow could be treated as spam; so a Webmaster who is buying links needs to know of the existence of rel=nofollow in order to avoid spamming.
I can see why rel=nofollow was felt necessary, and I agree it's a good idea in certain circumstances (particularly when built into software such as Wordpress, rather than expecting individual publishers to know about it and apply it). What I can't understand is why Google is now asking publishers to label paid links with rel=nofollow. The labelling of paid links was never mentioned as an application when rel=nofollow was introduced. There are some big problems with this approach:
- Even when rel=nofollow was introduced the Web was over 10 years old. What about all the old links, that were made before rel=nofollow existed? Are publishers supposed to go back and change them? All of them?
- Since rel=nofollow was introduced, it has become relatively well known in the dedicated search marketing and blogging community. But what efforts have been made to make it known to everyday Web designers and publishers?
- What exactly constitutes a paid link? A link to a parent company? A link to a partner company who has supplied you work in the past? What about a link to somebody who bought you a beer at a conference once? Or is it only a link for which money specifically was exchanged? In some commercial areas of the Web, depending on your point of view, all links could be considered paid. Although I don't work in the pills/porn/casino industries, even categories such as financial services are so highly commercialised that almost all links could be considered as paid. Within such a category, if all links were labelled with rel=nofollow (as Google appears to want), what actual use would rel=nofollow be?
Given the above problems, I can't quite work out whether Google is being humble or arrogant in asking publishers to label their links...
- Humble: It's like Google is admitting that it can no longer detect and properly compensate for paid links algorithmically. The heart of their ranking technology is links, and link spam is hurting so badly that they are asking for help.
- Arrogant: It's like Google is starting to lay down how the Web will be built. Rather than reacting to how people build Web sites, they are telling people how to build Web sites.
I am not enamoured with either of these possibilities.