View Single Post


Old
  Post #9 (permalink)   04-08-2007, 09:46 PM
Jayke
HD Newbie
 
Join Date: Jan 2007
Posts: 13

Status: Jayke is offline
Quote:
Originally Posted by ANMMark
Well the majority of search engines do automated crawling. Google has always followed the meta expire tag for us, and has revisited daily, when we have used:

<META HTTP-EQUIV="EXPIRES"
CONTENT="Mon, 30 Mar 1976 11:12:01 GMT">

Using a date in the past tells the search spider that the document has expired, and to schedule another visit on the next day.

I should add however that some engines will simply remove expired pages from their index.

The best way to keep this from happening is to generate your expire meta tag with a server-side language that updates the date to tomorrow's date every day, or with javascript that does the same. Server-side scripting is the best bet however, as it creates a static source for the spider to read.
Interesting about the META expires content. That shouldn't be too hard to do in PHP, thanks =]