Go Back   Web Design & SEO Company > SEO

SEO Search Engine Optimization, this section lists articles and tutorials on Search Engine Optimization for various Search Engines including Google, Yahoo & MSN. We provide SEO hints, tips and other free goodies to help you optimize your site and to start ranking well in the Search Engines.

Reply
 
  #1  
Old 04-22-2010, 12:16 PM
Junior Member
 
Join Date: Oct 2009
Posts: 6
Default Can anyone help?

Please can anyone explain me what is robots.txt file and its correct format?
Reply With Quote
Top SEO Tool
Harvester and Mass Blog Commenter
Blog Comment Software

  #2  
Old 04-27-2010, 09:49 AM
Senior Member
 
Join Date: Apr 2008
Posts: 164
Default

Website owners use the robots.txt file to give instructions about their site to web robots.
It works likes this: a robot wants to visit URL, but before it does so, it first checks for http://www......../robots.txt,

you got it?

I guess it's called The Robots Exclusion Protocol, you can google it)
Reply With Quote
  #3  
Old 04-28-2010, 09:23 AM
Member
 
Join Date: Jul 2008
Posts: 89
Default

It is a file you put on your site to tell search robots which pages you would like them not to visit.
Reply With Quote
  #4  
Old 05-21-2010, 08:03 AM
Senior Member
 
Join Date: Jun 2009
Posts: 222
Default

Robots.txt is a text (not html) file you put on your site to tell search robots which pages you would like them not to visit.
Reply With Quote
  #5  
Old 09-18-2010, 07:55 AM
Senior Member
 
Join Date: Aug 2009
Posts: 164
Default

robots.txt instruct the crawler not to access the page...... its main tactic like index, follow, no index, nofollow...
Reply With Quote
  #6  
Old 09-23-2010, 06:05 PM
Junior Member
 
Join Date: May 2009
Location: Unit 5, G/F Landco Corporate Center, Bajada, Davao City, Philippines
Posts: 18
Default

regarding its format..

The file itself is a simple text file, which can be created in Notepad. It need to be saved to the root directory of your site, that is the directory where your home page or index page is.
Reply With Quote
Top SEO Tool
Harvester and Mass Blog Commenter
Blog Comment Software

SEO - Top



Web Design & SEO Forums