Quote:
|
Originally Posted by pmhoran
UPDATE ... for those interested
I just about croaked when I saw what I saw ... For the month of November Inktomi Slurp had 116,409 spiders visit the site and total bandwidth used up was 1.01 GB
|
EQWebHost .... maybe you misunderstood ... but this 1.01 GB was bandwidth used JUST by the Inktomi Slurp spiders. In one month.
Or maybe my idea of "significant bandwidth usage" is different than yours.
The next closest bandwidth usage for the month by a spider/robot was for Googlebot ... and it used about 65 mb for the month. THAT I can deal with.
But 1.01 gigabytes??? That is just plain ridiculous.
Peter