View Single Post


Old
  Post #7 (permalink)   03-30-2007, 04:55 PM
ldcdc
HD Moderator
 
ldcdc's Avatar
 
Join Date: May 2004
Location: Ploiesti
Posts: 2,436

Status: ldcdc is offline
Quote:
You can add meta code to the top of your pages, to get the search engine to crawl the site and pages daily, regardless if you've updated them or not.
AFAIR there is a way to tell the SE not to crawl the site more often than <insert time period here>, but not a way to dictate a search engine how often to crawl it. At the end of the day, that's entirely to their discretion.

All this can be accomplished using the robots.txt file: http://en.wikipedia.org/wiki/Robots.txt