|
Most of these spiders are worthless and they do use bandwidth and system resources.
They are not giving you any more "people finding your website" they are just repetitively accessing your pages.
If you have a custom 404 page for instance that is fairly large this can use up a ton of resourses.
|