01536 316100

Blog Posts About : Robots.txt

a robots.txt equivalent to rel=canonical

In my last post I looked at the rel=canonical tag and finished by promising to look at some of the limitations of rel=canonical and consider some alternatives. Read More...

When should "NOINDEX" mean "INDEX"?

Matt Cutts has stirred up a little hornets' nest with his "What should NOINDEX do?" post. Matt reckons the topic will be colossally boring to some people - but not to me. For some reason I find Robots standards fascinating. Yep, I know I'm weird. Read More...

Robogenic - the one word expression for “search engine friendly”

The question is whether Google should completely drop a NOINDEX’ed page from our search results vs. show a reference to the page, or something in between? Read More...

The robots.txt file and the robots meta tag

This article was first written and published by me in 2000/2001, but no longer exists on the Web. It's still accurate - although search engines (notably Google) have taken steps to correct some of the problems described below, they can and do still arise. Read More...

Bringing Down Google With Two Simple Lines of Code

Is Google too powerful? It's a question asked by many. But much of Google's future depends on two simple lines of code. Read More...