Robots.txt Blog Posts

Your robots.txt file is an important part of your website. Learn more about it in our blog posts below.

Blogs about Robotstxt

blog
In my last post I looked at the rel=canonical tag and finished by promising to look at some of the limitations of rel=canonical and consider some alternatives.
By:
Alan Perkins
On: 8th March 2009
Posted In:
SEO | Google | Search Engines | Technical Architecture | Crawling and Indexing | Robots.txt
blog
Matt Cutts has stirred up a little hornets' nest with his "What should NOINDEX do?" post. Matt reckons the topic will be colossally boring to some people - but not to me. For some reason I find Robots standards fascinating. Yep, I know I'm weird.
By:
Alan Perkins
On: 29th February 2008
Posted In:
SEO | Google | Search Engines | Technical Architecture | Crawling and Indexing | Robots.txt
blog
The question is whether Google should completely drop a NOINDEX’ed page from our search results vs. show a reference to the page, or something in between?
By:
Alan Perkins
On: 19th October 2007
Posted In:
SEO | Google | Technical Architecture | Crawling and Indexing | Robots.txt
blog
This article was first written and published by me in 2000/2001, but no longer exists on the Web. It's still accurate - although search engines (notably Google) have taken steps to correct some of the problems described below, they can and do still arise.
By:
Alan Perkins
On: 18th September 2007
Posted In:
SEO | Google | Search Engines | Crawling and Indexing | Robots.txt
blog
Is Google too powerful? It's a question asked by many. But much of Google's future depends on two simple lines of code. When Google floated, its SEC Filing listed many potential future threats to its...
By:
Alan Perkins
On: 27th April 2007
Posted In:
SEO | Google | Search Engines | Robots.txt

Why Choose SilverDisc

Track Record Of Success

As a digital marketing agency with a 25-year history, continuously getting it right for our clients is of paramount importance and we have some great references to back this up.

Credibility

Being Google Premier Partners and Bing Select Partners gives you the assurance that we’re fully trained, that we have demonstrated exceptional account management across a range of client spends and industries, and that our track record will continue.

Thought Leaders

We see where digital marketing is heading and position our clients to meet it. For example, we are so far ahead of the SEO game that we helped Google to write their webmaster guidelines, so you can enjoy SEO success without fear of being penalised.

Ethical & Honest

We have your best interests at heart and we give it to you straight, even if that’s painful to you or us. For example, we won’t spend your money marketing a website when it’s plain to us that the money would be wasted. We look for a win-win-win …

Focused

We listen to your individual needs, identify targets and work relentlessly towards meeting them, concentrating on what’s effective.

Driven

We exist to unlock the value of technology for business, and we love what we do. We have 25 straight years in digital marketing.

Friendly, Open and Giving

These words are at the core of our values. We build strong partnerships and go out of our way to help solve problems for clients, suppliers and the wider world.

Let's Get Started

Please fill in the form below or give us a call on 01536 316100

Follow SilverDisc