Best Practice robots.txt

I did some research month ago and wrote a longer article about robots.txt. But for a quick lookup it’s too much to read, so today it’s time a for a simple best-practices check-list:

Read On: Encoding of the robots.txt file (20 Jul 2010; by hakre)
Image Credits: Robot © Christopher Grant Harris, Japanese Nada Letter provided by Haruno Akiha.

About these ads
This entry was posted in Pressed, Surviving the Internet and tagged , , , , , . Bookmark the permalink.

2 Responses to Best Practice robots.txt

  1. Pingback: How-To Block Alexa Robot | hakre on wordpress

  2. Mary Luketich says:

    Hi, I’m Mary Luketich and I can suggest the following URL: http://www.seomoz.org/learn-seo/robotstxt (It has nothing to do with encoding in robots.txt so it’s mainly unrelated).

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s